Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.X...X.......X.....X........................................] duration:
830.177808ms
FAIL (2025-08-18 10:23:54 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 830.206117ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [..X...X.......X.....X.......................................] duration:
1.012480787s
FAIL (2025-08-18 10:23:35 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.012517881s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [....................X.........X...............X.............] duration:
2.538936244s
FAIL (2025-08-18 10:23:25 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 18 10:12:28.814497391 (232h46m50.773594644s) [upload=242264]: Starting upload
Aug 18 10:12:28.814568850 (232h46m50.773666099s) [upload=242264]: Done starting upload
Aug 18 10:12:28.814652939 (232h46m50.773750187s) [upload=242264]: Uploading segments...
Aug 18 10:12:28.814925344 (232h46m50.77402259s) [upload=242264]: Got next segment. Inline: false
Aug 18 10:12:28.814972606 (232h46m50.774069851s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 18 10:12:28.814977665 (232h46m50.774074911s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 18 10:12:29.014812974 (232h46m50.973910208s) [seg_pos={0 0}, upload=242264]: Begin upload segment...
Aug 18 10:12:29.014870782 (232h46m50.973968016s) [seg_pos={0 0}, upload=242264]: Done begin upload segment.
Aug 18 10:12:29.014928044 (232h46m50.974025289s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.014949880 (232h46m50.974047113s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015381585 (232h46m50.974478838s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015505267 (232h46m50.974602528s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015688171 (232h46m50.974785417s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015899786 (232h46m50.97499704s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015982477 (232h46m50.975079722s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016282151 (232h46m50.975379399s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016313105 (232h46m50.975410357s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016584018 (232h46m50.975681263s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016879991 (232h46m50.975977246s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.017677212 (232h46m50.976774469s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018244920 (232h46m50.977342168s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018405683 (232h46m50.977502923s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018700697 (232h46m50.977797953s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018936705 (232h46m50.978033954s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019352075 (232h46m50.978449313s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019396481 (232h46m50.978493728s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.579861974 (232h46m51.538959224s) [upload=242264]: Next returned nil segment
Aug 18 10:12:29.579881340 (232h46m51.538978599s) [upload=242264]: Waiting for error group managing segments...
Aug 18 10:12:29.712502827 (232h46m51.671600067s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.715208093 (232h46m51.674305348s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.724656470 (232h46m51.683753719s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.731762008 (232h46m51.690859259s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.734744520 (232h46m51.693841759s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.735571428 (232h46m51.694668675s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.743776476 (232h46m51.702873735s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.755185797 (232h46m51.714283052s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.757620693 (232h46m51.716717925s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.761051617 (232h46m51.720148856s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.762461116 (232h46m51.721558363s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.779979392 (232h46m51.739076639s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.790560077 (232h46m51.749657337s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805160589 (232h46m51.764257838s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805207627 (232h46m51.764304874s) [seg_pos={0 0}, upload=242264]: Segment reached optimal threshold of 14 pieces.
Aug 18 10:12:29.805563963 (232h46m51.764661212s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WutpFSoqFPT4wPcAAsjsZD5WZy8boff1HmVefLHLyTfKNndy6): protocol: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection
Aug 18 10:12:29.805655820 (232h46m51.764753051s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection
Aug 18 10:12:29.805663766 (232h46m51.764761018s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Lo4tfpgtTeQbcNqfCb3zGHjEdRJUdf7hkThS9vfPgDvVLuEMW): protocol: expected piece hash; context canceled; EOF
Aug 18 10:12:29.805702123 (232h46m51.764799358s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gjesgEXSFr1aGyFgASrVRJgEHTne1KichuGqXwMjN1hHBt4kz): protocol: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection
Aug 18 10:12:29.805719533 (232h46m51.764816789s) [seg_pos={0 0}, upload=242264]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 18 10:12:29.805802398 (232h46m51.764899654s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 18 10:12:29.805809056 (232h46m51.764906311s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 18 10:12:29.954733853 (232h46m51.913831092s) [upload=242264]: Done uploading segments...
Aug 18 10:12:29.954776438 (232h46m51.913873679s) [upload=242264]: Upload finished. err: <nil>
Duration: 2.538975144s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [...X...X.......X.....X......................................] duration:
822.970659ms
FAIL (2025-08-18 10:23:13 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 822.999293ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [....X...X.......X.....X.....................................] duration:
841.703969ms
FAIL (2025-08-18 10:22:44 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 841.739413ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [.....................X.........X...............X............] duration:
2.408993282s
FAIL (2025-08-18 10:22:19 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 18 10:12:28.814497391 (232h46m50.773594644s) [upload=242264]: Starting upload
Aug 18 10:12:28.814568850 (232h46m50.773666099s) [upload=242264]: Done starting upload
Aug 18 10:12:28.814652939 (232h46m50.773750187s) [upload=242264]: Uploading segments...
Aug 18 10:12:28.814925344 (232h46m50.77402259s) [upload=242264]: Got next segment. Inline: false
Aug 18 10:12:28.814972606 (232h46m50.774069851s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 18 10:12:28.814977665 (232h46m50.774074911s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 18 10:12:29.014812974 (232h46m50.973910208s) [seg_pos={0 0}, upload=242264]: Begin upload segment...
Aug 18 10:12:29.014870782 (232h46m50.973968016s) [seg_pos={0 0}, upload=242264]: Done begin upload segment.
Aug 18 10:12:29.014928044 (232h46m50.974025289s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.014949880 (232h46m50.974047113s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015381585 (232h46m50.974478838s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015505267 (232h46m50.974602528s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015688171 (232h46m50.974785417s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015899786 (232h46m50.97499704s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015982477 (232h46m50.975079722s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016282151 (232h46m50.975379399s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016313105 (232h46m50.975410357s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016584018 (232h46m50.975681263s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016879991 (232h46m50.975977246s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.017677212 (232h46m50.976774469s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018244920 (232h46m50.977342168s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018405683 (232h46m50.977502923s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018700697 (232h46m50.977797953s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018936705 (232h46m50.978033954s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019352075 (232h46m50.978449313s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019396481 (232h46m50.978493728s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.579861974 (232h46m51.538959224s) [upload=242264]: Next returned nil segment
Aug 18 10:12:29.579881340 (232h46m51.538978599s) [upload=242264]: Waiting for error group managing segments...
Aug 18 10:12:29.712502827 (232h46m51.671600067s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.715208093 (232h46m51.674305348s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.724656470 (232h46m51.683753719s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.731762008 (232h46m51.690859259s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.734744520 (232h46m51.693841759s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.735571428 (232h46m51.694668675s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.743776476 (232h46m51.702873735s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.755185797 (232h46m51.714283052s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.757620693 (232h46m51.716717925s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.761051617 (232h46m51.720148856s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.762461116 (232h46m51.721558363s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.779979392 (232h46m51.739076639s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.790560077 (232h46m51.749657337s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805160589 (232h46m51.764257838s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805207627 (232h46m51.764304874s) [seg_pos={0 0}, upload=242264]: Segment reached optimal threshold of 14 pieces.
Aug 18 10:12:29.805563963 (232h46m51.764661212s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WutpFSoqFPT4wPcAAsjsZD5WZy8boff1HmVefLHLyTfKNndy6): protocol: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection
Aug 18 10:12:29.805655820 (232h46m51.764753051s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection
Aug 18 10:12:29.805663766 (232h46m51.764761018s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Lo4tfpgtTeQbcNqfCb3zGHjEdRJUdf7hkThS9vfPgDvVLuEMW): protocol: expected piece hash; context canceled; EOF
Aug 18 10:12:29.805702123 (232h46m51.764799358s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gjesgEXSFr1aGyFgASrVRJgEHTne1KichuGqXwMjN1hHBt4kz): protocol: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection
Aug 18 10:12:29.805719533 (232h46m51.764816789s) [seg_pos={0 0}, upload=242264]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 18 10:12:29.805802398 (232h46m51.764899654s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 18 10:12:29.805809056 (232h46m51.764906311s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 18 10:12:29.954733853 (232h46m51.913831092s) [upload=242264]: Done uploading segments...
Aug 18 10:12:29.954776438 (232h46m51.913873679s) [upload=242264]: Upload finished. err: <nil>
Duration: 2.409095699s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.....X...X.......X.....X....................................] duration:
1.407368098s
FAIL (2025-08-18 10:22:02 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.407496785s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [......................X.........X...............X...........] duration:
2.432701475s
FAIL (2025-08-18 10:21:27 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 18 10:12:28.814497391 (232h46m50.773594644s) [upload=242264]: Starting upload
Aug 18 10:12:28.814568850 (232h46m50.773666099s) [upload=242264]: Done starting upload
Aug 18 10:12:28.814652939 (232h46m50.773750187s) [upload=242264]: Uploading segments...
Aug 18 10:12:28.814925344 (232h46m50.77402259s) [upload=242264]: Got next segment. Inline: false
Aug 18 10:12:28.814972606 (232h46m50.774069851s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 18 10:12:28.814977665 (232h46m50.774074911s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 18 10:12:29.014812974 (232h46m50.973910208s) [seg_pos={0 0}, upload=242264]: Begin upload segment...
Aug 18 10:12:29.014870782 (232h46m50.973968016s) [seg_pos={0 0}, upload=242264]: Done begin upload segment.
Aug 18 10:12:29.014928044 (232h46m50.974025289s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.014949880 (232h46m50.974047113s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015381585 (232h46m50.974478838s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015505267 (232h46m50.974602528s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015688171 (232h46m50.974785417s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015899786 (232h46m50.97499704s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015982477 (232h46m50.975079722s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016282151 (232h46m50.975379399s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016313105 (232h46m50.975410357s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016584018 (232h46m50.975681263s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016879991 (232h46m50.975977246s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.017677212 (232h46m50.976774469s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018244920 (232h46m50.977342168s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018405683 (232h46m50.977502923s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018700697 (232h46m50.977797953s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018936705 (232h46m50.978033954s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019352075 (232h46m50.978449313s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019396481 (232h46m50.978493728s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.579861974 (232h46m51.538959224s) [upload=242264]: Next returned nil segment
Aug 18 10:12:29.579881340 (232h46m51.538978599s) [upload=242264]: Waiting for error group managing segments...
Aug 18 10:12:29.712502827 (232h46m51.671600067s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.715208093 (232h46m51.674305348s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.724656470 (232h46m51.683753719s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.731762008 (232h46m51.690859259s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.734744520 (232h46m51.693841759s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.735571428 (232h46m51.694668675s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.743776476 (232h46m51.702873735s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.755185797 (232h46m51.714283052s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.757620693 (232h46m51.716717925s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.761051617 (232h46m51.720148856s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.762461116 (232h46m51.721558363s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.779979392 (232h46m51.739076639s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.790560077 (232h46m51.749657337s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805160589 (232h46m51.764257838s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805207627 (232h46m51.764304874s) [seg_pos={0 0}, upload=242264]: Segment reached optimal threshold of 14 pieces.
Aug 18 10:12:29.805563963 (232h46m51.764661212s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WutpFSoqFPT4wPcAAsjsZD5WZy8boff1HmVefLHLyTfKNndy6): protocol: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection
Aug 18 10:12:29.805655820 (232h46m51.764753051s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection
Aug 18 10:12:29.805663766 (232h46m51.764761018s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Lo4tfpgtTeQbcNqfCb3zGHjEdRJUdf7hkThS9vfPgDvVLuEMW): protocol: expected piece hash; context canceled; EOF
Aug 18 10:12:29.805702123 (232h46m51.764799358s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gjesgEXSFr1aGyFgASrVRJgEHTne1KichuGqXwMjN1hHBt4kz): protocol: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection
Aug 18 10:12:29.805719533 (232h46m51.764816789s) [seg_pos={0 0}, upload=242264]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 18 10:12:29.805802398 (232h46m51.764899654s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 18 10:12:29.805809056 (232h46m51.764906311s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 18 10:12:29.954733853 (232h46m51.913831092s) [upload=242264]: Done uploading segments...
Aug 18 10:12:29.954776438 (232h46m51.913873679s) [upload=242264]: Upload finished. err: <nil>
Duration: 2.432747917s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [......X...X.......X.....X...................................] duration:
840.471772ms
FAIL (2025-08-18 10:21:06 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 840.503311ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [.......................X.........X...............X..........] duration:
2.358259847s
FAIL (2025-08-18 10:20:59 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 18 10:12:28.814497391 (232h46m50.773594644s) [upload=242264]: Starting upload
Aug 18 10:12:28.814568850 (232h46m50.773666099s) [upload=242264]: Done starting upload
Aug 18 10:12:28.814652939 (232h46m50.773750187s) [upload=242264]: Uploading segments...
Aug 18 10:12:28.814925344 (232h46m50.77402259s) [upload=242264]: Got next segment. Inline: false
Aug 18 10:12:28.814972606 (232h46m50.774069851s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 18 10:12:28.814977665 (232h46m50.774074911s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 18 10:12:29.014812974 (232h46m50.973910208s) [seg_pos={0 0}, upload=242264]: Begin upload segment...
Aug 18 10:12:29.014870782 (232h46m50.973968016s) [seg_pos={0 0}, upload=242264]: Done begin upload segment.
Aug 18 10:12:29.014928044 (232h46m50.974025289s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.014949880 (232h46m50.974047113s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015381585 (232h46m50.974478838s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015505267 (232h46m50.974602528s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015688171 (232h46m50.974785417s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015899786 (232h46m50.97499704s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015982477 (232h46m50.975079722s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016282151 (232h46m50.975379399s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016313105 (232h46m50.975410357s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016584018 (232h46m50.975681263s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016879991 (232h46m50.975977246s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.017677212 (232h46m50.976774469s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018244920 (232h46m50.977342168s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018405683 (232h46m50.977502923s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018700697 (232h46m50.977797953s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018936705 (232h46m50.978033954s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019352075 (232h46m50.978449313s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019396481 (232h46m50.978493728s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.579861974 (232h46m51.538959224s) [upload=242264]: Next returned nil segment
Aug 18 10:12:29.579881340 (232h46m51.538978599s) [upload=242264]: Waiting for error group managing segments...
Aug 18 10:12:29.712502827 (232h46m51.671600067s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.715208093 (232h46m51.674305348s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.724656470 (232h46m51.683753719s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.731762008 (232h46m51.690859259s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.734744520 (232h46m51.693841759s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.735571428 (232h46m51.694668675s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.743776476 (232h46m51.702873735s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.755185797 (232h46m51.714283052s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.757620693 (232h46m51.716717925s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.761051617 (232h46m51.720148856s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.762461116 (232h46m51.721558363s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.779979392 (232h46m51.739076639s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.790560077 (232h46m51.749657337s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805160589 (232h46m51.764257838s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805207627 (232h46m51.764304874s) [seg_pos={0 0}, upload=242264]: Segment reached optimal threshold of 14 pieces.
Aug 18 10:12:29.805563963 (232h46m51.764661212s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WutpFSoqFPT4wPcAAsjsZD5WZy8boff1HmVefLHLyTfKNndy6): protocol: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection
Aug 18 10:12:29.805655820 (232h46m51.764753051s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection
Aug 18 10:12:29.805663766 (232h46m51.764761018s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Lo4tfpgtTeQbcNqfCb3zGHjEdRJUdf7hkThS9vfPgDvVLuEMW): protocol: expected piece hash; context canceled; EOF
Aug 18 10:12:29.805702123 (232h46m51.764799358s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gjesgEXSFr1aGyFgASrVRJgEHTne1KichuGqXwMjN1hHBt4kz): protocol: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection
Aug 18 10:12:29.805719533 (232h46m51.764816789s) [seg_pos={0 0}, upload=242264]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 18 10:12:29.805802398 (232h46m51.764899654s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 18 10:12:29.805809056 (232h46m51.764906311s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 18 10:12:29.954733853 (232h46m51.913831092s) [upload=242264]: Done uploading segments...
Aug 18 10:12:29.954776438 (232h46m51.913873679s) [upload=242264]: Upload finished. err: <nil>
Duration: 2.358308921s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.......X...X.......X.....X..................................] duration:
854.372098ms
FAIL (2025-08-18 10:20:08 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 854.401913ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [........................X.........X...............X.........] duration:
2.666498903s
FAIL (2025-08-18 10:19:45 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 18 10:12:28.814497391 (232h46m50.773594644s) [upload=242264]: Starting upload
Aug 18 10:12:28.814568850 (232h46m50.773666099s) [upload=242264]: Done starting upload
Aug 18 10:12:28.814652939 (232h46m50.773750187s) [upload=242264]: Uploading segments...
Aug 18 10:12:28.814925344 (232h46m50.77402259s) [upload=242264]: Got next segment. Inline: false
Aug 18 10:12:28.814972606 (232h46m50.774069851s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 18 10:12:28.814977665 (232h46m50.774074911s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 18 10:12:29.014812974 (232h46m50.973910208s) [seg_pos={0 0}, upload=242264]: Begin upload segment...
Aug 18 10:12:29.014870782 (232h46m50.973968016s) [seg_pos={0 0}, upload=242264]: Done begin upload segment.
Aug 18 10:12:29.014928044 (232h46m50.974025289s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.014949880 (232h46m50.974047113s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015381585 (232h46m50.974478838s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015505267 (232h46m50.974602528s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015688171 (232h46m50.974785417s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015899786 (232h46m50.97499704s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015982477 (232h46m50.975079722s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016282151 (232h46m50.975379399s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016313105 (232h46m50.975410357s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016584018 (232h46m50.975681263s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016879991 (232h46m50.975977246s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.017677212 (232h46m50.976774469s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018244920 (232h46m50.977342168s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018405683 (232h46m50.977502923s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018700697 (232h46m50.977797953s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018936705 (232h46m50.978033954s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019352075 (232h46m50.978449313s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019396481 (232h46m50.978493728s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.579861974 (232h46m51.538959224s) [upload=242264]: Next returned nil segment
Aug 18 10:12:29.579881340 (232h46m51.538978599s) [upload=242264]: Waiting for error group managing segments...
Aug 18 10:12:29.712502827 (232h46m51.671600067s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.715208093 (232h46m51.674305348s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.724656470 (232h46m51.683753719s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.731762008 (232h46m51.690859259s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.734744520 (232h46m51.693841759s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.735571428 (232h46m51.694668675s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.743776476 (232h46m51.702873735s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.755185797 (232h46m51.714283052s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.757620693 (232h46m51.716717925s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.761051617 (232h46m51.720148856s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.762461116 (232h46m51.721558363s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.779979392 (232h46m51.739076639s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.790560077 (232h46m51.749657337s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805160589 (232h46m51.764257838s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805207627 (232h46m51.764304874s) [seg_pos={0 0}, upload=242264]: Segment reached optimal threshold of 14 pieces.
Aug 18 10:12:29.805563963 (232h46m51.764661212s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WutpFSoqFPT4wPcAAsjsZD5WZy8boff1HmVefLHLyTfKNndy6): protocol: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection
Aug 18 10:12:29.805655820 (232h46m51.764753051s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection
Aug 18 10:12:29.805663766 (232h46m51.764761018s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Lo4tfpgtTeQbcNqfCb3zGHjEdRJUdf7hkThS9vfPgDvVLuEMW): protocol: expected piece hash; context canceled; EOF
Aug 18 10:12:29.805702123 (232h46m51.764799358s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gjesgEXSFr1aGyFgASrVRJgEHTne1KichuGqXwMjN1hHBt4kz): protocol: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection
Aug 18 10:12:29.805719533 (232h46m51.764816789s) [seg_pos={0 0}, upload=242264]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 18 10:12:29.805802398 (232h46m51.764899654s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 18 10:12:29.805809056 (232h46m51.764906311s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 18 10:12:29.954733853 (232h46m51.913831092s) [upload=242264]: Done uploading segments...
Aug 18 10:12:29.954776438 (232h46m51.913873679s) [upload=242264]: Upload finished. err: <nil>
Duration: 2.666552633s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [.........................X.........X...............X........] duration:
2.443109584s
FAIL (2025-08-18 10:19:11 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 18 10:12:28.814497391 (232h46m50.773594644s) [upload=242264]: Starting upload
Aug 18 10:12:28.814568850 (232h46m50.773666099s) [upload=242264]: Done starting upload
Aug 18 10:12:28.814652939 (232h46m50.773750187s) [upload=242264]: Uploading segments...
Aug 18 10:12:28.814925344 (232h46m50.77402259s) [upload=242264]: Got next segment. Inline: false
Aug 18 10:12:28.814972606 (232h46m50.774069851s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 18 10:12:28.814977665 (232h46m50.774074911s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 18 10:12:29.014812974 (232h46m50.973910208s) [seg_pos={0 0}, upload=242264]: Begin upload segment...
Aug 18 10:12:29.014870782 (232h46m50.973968016s) [seg_pos={0 0}, upload=242264]: Done begin upload segment.
Aug 18 10:12:29.014928044 (232h46m50.974025289s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.014949880 (232h46m50.974047113s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015381585 (232h46m50.974478838s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015505267 (232h46m50.974602528s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015688171 (232h46m50.974785417s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015899786 (232h46m50.97499704s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015982477 (232h46m50.975079722s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016282151 (232h46m50.975379399s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016313105 (232h46m50.975410357s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016584018 (232h46m50.975681263s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016879991 (232h46m50.975977246s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.017677212 (232h46m50.976774469s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018244920 (232h46m50.977342168s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018405683 (232h46m50.977502923s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018700697 (232h46m50.977797953s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018936705 (232h46m50.978033954s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019352075 (232h46m50.978449313s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019396481 (232h46m50.978493728s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.579861974 (232h46m51.538959224s) [upload=242264]: Next returned nil segment
Aug 18 10:12:29.579881340 (232h46m51.538978599s) [upload=242264]: Waiting for error group managing segments...
Aug 18 10:12:29.712502827 (232h46m51.671600067s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.715208093 (232h46m51.674305348s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.724656470 (232h46m51.683753719s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.731762008 (232h46m51.690859259s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.734744520 (232h46m51.693841759s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.735571428 (232h46m51.694668675s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.743776476 (232h46m51.702873735s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.755185797 (232h46m51.714283052s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.757620693 (232h46m51.716717925s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.761051617 (232h46m51.720148856s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.762461116 (232h46m51.721558363s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.779979392 (232h46m51.739076639s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.790560077 (232h46m51.749657337s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805160589 (232h46m51.764257838s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805207627 (232h46m51.764304874s) [seg_pos={0 0}, upload=242264]: Segment reached optimal threshold of 14 pieces.
Aug 18 10:12:29.805563963 (232h46m51.764661212s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WutpFSoqFPT4wPcAAsjsZD5WZy8boff1HmVefLHLyTfKNndy6): protocol: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection
Aug 18 10:12:29.805655820 (232h46m51.764753051s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection
Aug 18 10:12:29.805663766 (232h46m51.764761018s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Lo4tfpgtTeQbcNqfCb3zGHjEdRJUdf7hkThS9vfPgDvVLuEMW): protocol: expected piece hash; context canceled; EOF
Aug 18 10:12:29.805702123 (232h46m51.764799358s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gjesgEXSFr1aGyFgASrVRJgEHTne1KichuGqXwMjN1hHBt4kz): protocol: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection
Aug 18 10:12:29.805719533 (232h46m51.764816789s) [seg_pos={0 0}, upload=242264]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 18 10:12:29.805802398 (232h46m51.764899654s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 18 10:12:29.805809056 (232h46m51.764906311s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 18 10:12:29.954733853 (232h46m51.913831092s) [upload=242264]: Done uploading segments...
Aug 18 10:12:29.954776438 (232h46m51.913873679s) [upload=242264]: Upload finished. err: <nil>
Duration: 2.443154971s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [........X...X.......X.....X.................................] duration:
965.85183ms
FAIL (2025-08-18 10:19:11 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 965.882046ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.........X...X.......X.....X................................] duration:
817.935069ms
FAIL (2025-08-18 10:18:22 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 818.02904ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [..........................X.........X...............X.......] duration:
2.360914068s
FAIL (2025-08-18 10:18:19 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 18 10:12:28.814497391 (232h46m50.773594644s) [upload=242264]: Starting upload
Aug 18 10:12:28.814568850 (232h46m50.773666099s) [upload=242264]: Done starting upload
Aug 18 10:12:28.814652939 (232h46m50.773750187s) [upload=242264]: Uploading segments...
Aug 18 10:12:28.814925344 (232h46m50.77402259s) [upload=242264]: Got next segment. Inline: false
Aug 18 10:12:28.814972606 (232h46m50.774069851s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 18 10:12:28.814977665 (232h46m50.774074911s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 18 10:12:29.014812974 (232h46m50.973910208s) [seg_pos={0 0}, upload=242264]: Begin upload segment...
Aug 18 10:12:29.014870782 (232h46m50.973968016s) [seg_pos={0 0}, upload=242264]: Done begin upload segment.
Aug 18 10:12:29.014928044 (232h46m50.974025289s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.014949880 (232h46m50.974047113s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015381585 (232h46m50.974478838s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015505267 (232h46m50.974602528s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015688171 (232h46m50.974785417s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015899786 (232h46m50.97499704s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015982477 (232h46m50.975079722s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016282151 (232h46m50.975379399s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016313105 (232h46m50.975410357s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016584018 (232h46m50.975681263s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016879991 (232h46m50.975977246s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.017677212 (232h46m50.976774469s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018244920 (232h46m50.977342168s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018405683 (232h46m50.977502923s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018700697 (232h46m50.977797953s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018936705 (232h46m50.978033954s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019352075 (232h46m50.978449313s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019396481 (232h46m50.978493728s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.579861974 (232h46m51.538959224s) [upload=242264]: Next returned nil segment
Aug 18 10:12:29.579881340 (232h46m51.538978599s) [upload=242264]: Waiting for error group managing segments...
Aug 18 10:12:29.712502827 (232h46m51.671600067s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.715208093 (232h46m51.674305348s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.724656470 (232h46m51.683753719s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.731762008 (232h46m51.690859259s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.734744520 (232h46m51.693841759s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.735571428 (232h46m51.694668675s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.743776476 (232h46m51.702873735s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.755185797 (232h46m51.714283052s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.757620693 (232h46m51.716717925s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.761051617 (232h46m51.720148856s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.762461116 (232h46m51.721558363s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.779979392 (232h46m51.739076639s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.790560077 (232h46m51.749657337s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805160589 (232h46m51.764257838s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805207627 (232h46m51.764304874s) [seg_pos={0 0}, upload=242264]: Segment reached optimal threshold of 14 pieces.
Aug 18 10:12:29.805563963 (232h46m51.764661212s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WutpFSoqFPT4wPcAAsjsZD5WZy8boff1HmVefLHLyTfKNndy6): protocol: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection
Aug 18 10:12:29.805655820 (232h46m51.764753051s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection
Aug 18 10:12:29.805663766 (232h46m51.764761018s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Lo4tfpgtTeQbcNqfCb3zGHjEdRJUdf7hkThS9vfPgDvVLuEMW): protocol: expected piece hash; context canceled; EOF
Aug 18 10:12:29.805702123 (232h46m51.764799358s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gjesgEXSFr1aGyFgASrVRJgEHTne1KichuGqXwMjN1hHBt4kz): protocol: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection
Aug 18 10:12:29.805719533 (232h46m51.764816789s) [seg_pos={0 0}, upload=242264]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 18 10:12:29.805802398 (232h46m51.764899654s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 18 10:12:29.805809056 (232h46m51.764906311s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 18 10:12:29.954733853 (232h46m51.913831092s) [upload=242264]: Done uploading segments...
Aug 18 10:12:29.954776438 (232h46m51.913873679s) [upload=242264]: Upload finished. err: <nil>
Duration: 2.360988552s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [...........................X.........X...............X......] duration:
2.830024917s
FAIL (2025-08-18 10:17:34 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 18 10:12:28.814497391 (232h46m50.773594644s) [upload=242264]: Starting upload
Aug 18 10:12:28.814568850 (232h46m50.773666099s) [upload=242264]: Done starting upload
Aug 18 10:12:28.814652939 (232h46m50.773750187s) [upload=242264]: Uploading segments...
Aug 18 10:12:28.814925344 (232h46m50.77402259s) [upload=242264]: Got next segment. Inline: false
Aug 18 10:12:28.814972606 (232h46m50.774069851s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 18 10:12:28.814977665 (232h46m50.774074911s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 18 10:12:29.014812974 (232h46m50.973910208s) [seg_pos={0 0}, upload=242264]: Begin upload segment...
Aug 18 10:12:29.014870782 (232h46m50.973968016s) [seg_pos={0 0}, upload=242264]: Done begin upload segment.
Aug 18 10:12:29.014928044 (232h46m50.974025289s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.014949880 (232h46m50.974047113s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015381585 (232h46m50.974478838s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015505267 (232h46m50.974602528s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015688171 (232h46m50.974785417s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015899786 (232h46m50.97499704s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015982477 (232h46m50.975079722s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016282151 (232h46m50.975379399s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016313105 (232h46m50.975410357s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016584018 (232h46m50.975681263s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016879991 (232h46m50.975977246s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.017677212 (232h46m50.976774469s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018244920 (232h46m50.977342168s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018405683 (232h46m50.977502923s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018700697 (232h46m50.977797953s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018936705 (232h46m50.978033954s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019352075 (232h46m50.978449313s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019396481 (232h46m50.978493728s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.579861974 (232h46m51.538959224s) [upload=242264]: Next returned nil segment
Aug 18 10:12:29.579881340 (232h46m51.538978599s) [upload=242264]: Waiting for error group managing segments...
Aug 18 10:12:29.712502827 (232h46m51.671600067s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.715208093 (232h46m51.674305348s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.724656470 (232h46m51.683753719s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.731762008 (232h46m51.690859259s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.734744520 (232h46m51.693841759s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.735571428 (232h46m51.694668675s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.743776476 (232h46m51.702873735s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.755185797 (232h46m51.714283052s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.757620693 (232h46m51.716717925s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.761051617 (232h46m51.720148856s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.762461116 (232h46m51.721558363s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.779979392 (232h46m51.739076639s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.790560077 (232h46m51.749657337s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805160589 (232h46m51.764257838s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805207627 (232h46m51.764304874s) [seg_pos={0 0}, upload=242264]: Segment reached optimal threshold of 14 pieces.
Aug 18 10:12:29.805563963 (232h46m51.764661212s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WutpFSoqFPT4wPcAAsjsZD5WZy8boff1HmVefLHLyTfKNndy6): protocol: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection
Aug 18 10:12:29.805655820 (232h46m51.764753051s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection
Aug 18 10:12:29.805663766 (232h46m51.764761018s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Lo4tfpgtTeQbcNqfCb3zGHjEdRJUdf7hkThS9vfPgDvVLuEMW): protocol: expected piece hash; context canceled; EOF
Aug 18 10:12:29.805702123 (232h46m51.764799358s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gjesgEXSFr1aGyFgASrVRJgEHTne1KichuGqXwMjN1hHBt4kz): protocol: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection
Aug 18 10:12:29.805719533 (232h46m51.764816789s) [seg_pos={0 0}, upload=242264]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 18 10:12:29.805802398 (232h46m51.764899654s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 18 10:12:29.805809056 (232h46m51.764906311s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 18 10:12:29.954733853 (232h46m51.913831092s) [upload=242264]: Done uploading segments...
Aug 18 10:12:29.954776438 (232h46m51.913873679s) [upload=242264]: Upload finished. err: <nil>
Duration: 2.830064371s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [..........X...X.......X.....X...............................] duration:
911.982268ms
FAIL (2025-08-18 10:17:10 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 912.068197ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [............................X.........X...............X.....] duration:
2.362071497s
FAIL (2025-08-18 10:16:40 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 18 10:12:28.814497391 (232h46m50.773594644s) [upload=242264]: Starting upload
Aug 18 10:12:28.814568850 (232h46m50.773666099s) [upload=242264]: Done starting upload
Aug 18 10:12:28.814652939 (232h46m50.773750187s) [upload=242264]: Uploading segments...
Aug 18 10:12:28.814925344 (232h46m50.77402259s) [upload=242264]: Got next segment. Inline: false
Aug 18 10:12:28.814972606 (232h46m50.774069851s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 18 10:12:28.814977665 (232h46m50.774074911s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 18 10:12:29.014812974 (232h46m50.973910208s) [seg_pos={0 0}, upload=242264]: Begin upload segment...
Aug 18 10:12:29.014870782 (232h46m50.973968016s) [seg_pos={0 0}, upload=242264]: Done begin upload segment.
Aug 18 10:12:29.014928044 (232h46m50.974025289s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.014949880 (232h46m50.974047113s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015381585 (232h46m50.974478838s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015505267 (232h46m50.974602528s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015688171 (232h46m50.974785417s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015899786 (232h46m50.97499704s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015982477 (232h46m50.975079722s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016282151 (232h46m50.975379399s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016313105 (232h46m50.975410357s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016584018 (232h46m50.975681263s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016879991 (232h46m50.975977246s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.017677212 (232h46m50.976774469s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018244920 (232h46m50.977342168s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018405683 (232h46m50.977502923s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018700697 (232h46m50.977797953s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018936705 (232h46m50.978033954s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019352075 (232h46m50.978449313s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019396481 (232h46m50.978493728s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.579861974 (232h46m51.538959224s) [upload=242264]: Next returned nil segment
Aug 18 10:12:29.579881340 (232h46m51.538978599s) [upload=242264]: Waiting for error group managing segments...
Aug 18 10:12:29.712502827 (232h46m51.671600067s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.715208093 (232h46m51.674305348s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.724656470 (232h46m51.683753719s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.731762008 (232h46m51.690859259s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.734744520 (232h46m51.693841759s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.735571428 (232h46m51.694668675s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.743776476 (232h46m51.702873735s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.755185797 (232h46m51.714283052s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.757620693 (232h46m51.716717925s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.761051617 (232h46m51.720148856s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.762461116 (232h46m51.721558363s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.779979392 (232h46m51.739076639s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.790560077 (232h46m51.749657337s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805160589 (232h46m51.764257838s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805207627 (232h46m51.764304874s) [seg_pos={0 0}, upload=242264]: Segment reached optimal threshold of 14 pieces.
Aug 18 10:12:29.805563963 (232h46m51.764661212s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WutpFSoqFPT4wPcAAsjsZD5WZy8boff1HmVefLHLyTfKNndy6): protocol: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection
Aug 18 10:12:29.805655820 (232h46m51.764753051s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection
Aug 18 10:12:29.805663766 (232h46m51.764761018s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Lo4tfpgtTeQbcNqfCb3zGHjEdRJUdf7hkThS9vfPgDvVLuEMW): protocol: expected piece hash; context canceled; EOF
Aug 18 10:12:29.805702123 (232h46m51.764799358s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gjesgEXSFr1aGyFgASrVRJgEHTne1KichuGqXwMjN1hHBt4kz): protocol: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection
Aug 18 10:12:29.805719533 (232h46m51.764816789s) [seg_pos={0 0}, upload=242264]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 18 10:12:29.805802398 (232h46m51.764899654s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 18 10:12:29.805809056 (232h46m51.764906311s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 18 10:12:29.954733853 (232h46m51.913831092s) [upload=242264]: Done uploading segments...
Aug 18 10:12:29.954776438 (232h46m51.913873679s) [upload=242264]: Upload finished. err: <nil>
Duration: 2.362104567s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [...........X...X.......X.....X..............................] duration:
872.255218ms
FAIL (2025-08-18 10:16:22 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 872.284206ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [X............................X.........X...............X....] duration:
2.245555658s
FAIL (2025-08-18 10:15:54 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 18 10:12:28.814497391 (232h46m50.773594644s) [upload=242264]: Starting upload
Aug 18 10:12:28.814568850 (232h46m50.773666099s) [upload=242264]: Done starting upload
Aug 18 10:12:28.814652939 (232h46m50.773750187s) [upload=242264]: Uploading segments...
Aug 18 10:12:28.814925344 (232h46m50.77402259s) [upload=242264]: Got next segment. Inline: false
Aug 18 10:12:28.814972606 (232h46m50.774069851s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 18 10:12:28.814977665 (232h46m50.774074911s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 18 10:12:29.014812974 (232h46m50.973910208s) [seg_pos={0 0}, upload=242264]: Begin upload segment...
Aug 18 10:12:29.014870782 (232h46m50.973968016s) [seg_pos={0 0}, upload=242264]: Done begin upload segment.
Aug 18 10:12:29.014928044 (232h46m50.974025289s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.014949880 (232h46m50.974047113s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015381585 (232h46m50.974478838s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015505267 (232h46m50.974602528s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015688171 (232h46m50.974785417s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015899786 (232h46m50.97499704s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015982477 (232h46m50.975079722s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016282151 (232h46m50.975379399s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016313105 (232h46m50.975410357s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016584018 (232h46m50.975681263s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016879991 (232h46m50.975977246s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.017677212 (232h46m50.976774469s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018244920 (232h46m50.977342168s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018405683 (232h46m50.977502923s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018700697 (232h46m50.977797953s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018936705 (232h46m50.978033954s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019352075 (232h46m50.978449313s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019396481 (232h46m50.978493728s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.579861974 (232h46m51.538959224s) [upload=242264]: Next returned nil segment
Aug 18 10:12:29.579881340 (232h46m51.538978599s) [upload=242264]: Waiting for error group managing segments...
Aug 18 10:12:29.712502827 (232h46m51.671600067s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.715208093 (232h46m51.674305348s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.724656470 (232h46m51.683753719s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.731762008 (232h46m51.690859259s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.734744520 (232h46m51.693841759s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.735571428 (232h46m51.694668675s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.743776476 (232h46m51.702873735s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.755185797 (232h46m51.714283052s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.757620693 (232h46m51.716717925s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.761051617 (232h46m51.720148856s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.762461116 (232h46m51.721558363s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.779979392 (232h46m51.739076639s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.790560077 (232h46m51.749657337s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805160589 (232h46m51.764257838s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805207627 (232h46m51.764304874s) [seg_pos={0 0}, upload=242264]: Segment reached optimal threshold of 14 pieces.
Aug 18 10:12:29.805563963 (232h46m51.764661212s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WutpFSoqFPT4wPcAAsjsZD5WZy8boff1HmVefLHLyTfKNndy6): protocol: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection
Aug 18 10:12:29.805655820 (232h46m51.764753051s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection
Aug 18 10:12:29.805663766 (232h46m51.764761018s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Lo4tfpgtTeQbcNqfCb3zGHjEdRJUdf7hkThS9vfPgDvVLuEMW): protocol: expected piece hash; context canceled; EOF
Aug 18 10:12:29.805702123 (232h46m51.764799358s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gjesgEXSFr1aGyFgASrVRJgEHTne1KichuGqXwMjN1hHBt4kz): protocol: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection
Aug 18 10:12:29.805719533 (232h46m51.764816789s) [seg_pos={0 0}, upload=242264]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 18 10:12:29.805802398 (232h46m51.764899654s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 18 10:12:29.805809056 (232h46m51.764906311s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 18 10:12:29.954733853 (232h46m51.913831092s) [upload=242264]: Done uploading segments...
Aug 18 10:12:29.954776438 (232h46m51.913873679s) [upload=242264]: Upload finished. err: <nil>
Duration: 2.245587334s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [............X...X.......X.....X.............................] duration:
962.286657ms
FAIL (2025-08-18 10:15:45 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 962.316313ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [.X............................X.........X...............X...] duration:
2.256624986s
FAIL (2025-08-18 10:15:09 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 18 10:12:28.814497391 (232h46m50.773594644s) [upload=242264]: Starting upload
Aug 18 10:12:28.814568850 (232h46m50.773666099s) [upload=242264]: Done starting upload
Aug 18 10:12:28.814652939 (232h46m50.773750187s) [upload=242264]: Uploading segments...
Aug 18 10:12:28.814925344 (232h46m50.77402259s) [upload=242264]: Got next segment. Inline: false
Aug 18 10:12:28.814972606 (232h46m50.774069851s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 18 10:12:28.814977665 (232h46m50.774074911s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 18 10:12:29.014812974 (232h46m50.973910208s) [seg_pos={0 0}, upload=242264]: Begin upload segment...
Aug 18 10:12:29.014870782 (232h46m50.973968016s) [seg_pos={0 0}, upload=242264]: Done begin upload segment.
Aug 18 10:12:29.014928044 (232h46m50.974025289s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.014949880 (232h46m50.974047113s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015381585 (232h46m50.974478838s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015505267 (232h46m50.974602528s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015688171 (232h46m50.974785417s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015899786 (232h46m50.97499704s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015982477 (232h46m50.975079722s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016282151 (232h46m50.975379399s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016313105 (232h46m50.975410357s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016584018 (232h46m50.975681263s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016879991 (232h46m50.975977246s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.017677212 (232h46m50.976774469s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018244920 (232h46m50.977342168s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018405683 (232h46m50.977502923s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018700697 (232h46m50.977797953s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018936705 (232h46m50.978033954s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019352075 (232h46m50.978449313s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019396481 (232h46m50.978493728s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.579861974 (232h46m51.538959224s) [upload=242264]: Next returned nil segment
Aug 18 10:12:29.579881340 (232h46m51.538978599s) [upload=242264]: Waiting for error group managing segments...
Aug 18 10:12:29.712502827 (232h46m51.671600067s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.715208093 (232h46m51.674305348s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.724656470 (232h46m51.683753719s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.731762008 (232h46m51.690859259s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.734744520 (232h46m51.693841759s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.735571428 (232h46m51.694668675s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.743776476 (232h46m51.702873735s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.755185797 (232h46m51.714283052s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.757620693 (232h46m51.716717925s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.761051617 (232h46m51.720148856s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.762461116 (232h46m51.721558363s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.779979392 (232h46m51.739076639s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.790560077 (232h46m51.749657337s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805160589 (232h46m51.764257838s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805207627 (232h46m51.764304874s) [seg_pos={0 0}, upload=242264]: Segment reached optimal threshold of 14 pieces.
Aug 18 10:12:29.805563963 (232h46m51.764661212s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WutpFSoqFPT4wPcAAsjsZD5WZy8boff1HmVefLHLyTfKNndy6): protocol: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection
Aug 18 10:12:29.805655820 (232h46m51.764753051s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection
Aug 18 10:12:29.805663766 (232h46m51.764761018s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Lo4tfpgtTeQbcNqfCb3zGHjEdRJUdf7hkThS9vfPgDvVLuEMW): protocol: expected piece hash; context canceled; EOF
Aug 18 10:12:29.805702123 (232h46m51.764799358s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gjesgEXSFr1aGyFgASrVRJgEHTne1KichuGqXwMjN1hHBt4kz): protocol: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection
Aug 18 10:12:29.805719533 (232h46m51.764816789s) [seg_pos={0 0}, upload=242264]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 18 10:12:29.805802398 (232h46m51.764899654s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 18 10:12:29.805809056 (232h46m51.764906311s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 18 10:12:29.954733853 (232h46m51.913831092s) [upload=242264]: Done uploading segments...
Aug 18 10:12:29.954776438 (232h46m51.913873679s) [upload=242264]: Upload finished. err: <nil>
Duration: 2.256741337s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.............X...X.......X.....X............................] duration:
1.020289088s
FAIL (2025-08-18 10:14:40 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.020342798s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [..X............................X.........X...............X..] duration:
2.52170215s
FAIL (2025-08-18 10:14:27 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 18 10:12:28.814497391 (232h46m50.773594644s) [upload=242264]: Starting upload
Aug 18 10:12:28.814568850 (232h46m50.773666099s) [upload=242264]: Done starting upload
Aug 18 10:12:28.814652939 (232h46m50.773750187s) [upload=242264]: Uploading segments...
Aug 18 10:12:28.814925344 (232h46m50.77402259s) [upload=242264]: Got next segment. Inline: false
Aug 18 10:12:28.814972606 (232h46m50.774069851s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 18 10:12:28.814977665 (232h46m50.774074911s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 18 10:12:29.014812974 (232h46m50.973910208s) [seg_pos={0 0}, upload=242264]: Begin upload segment...
Aug 18 10:12:29.014870782 (232h46m50.973968016s) [seg_pos={0 0}, upload=242264]: Done begin upload segment.
Aug 18 10:12:29.014928044 (232h46m50.974025289s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.014949880 (232h46m50.974047113s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015381585 (232h46m50.974478838s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015505267 (232h46m50.974602528s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015688171 (232h46m50.974785417s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015899786 (232h46m50.97499704s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015982477 (232h46m50.975079722s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016282151 (232h46m50.975379399s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016313105 (232h46m50.975410357s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016584018 (232h46m50.975681263s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016879991 (232h46m50.975977246s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.017677212 (232h46m50.976774469s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018244920 (232h46m50.977342168s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018405683 (232h46m50.977502923s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018700697 (232h46m50.977797953s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018936705 (232h46m50.978033954s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019352075 (232h46m50.978449313s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019396481 (232h46m50.978493728s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.579861974 (232h46m51.538959224s) [upload=242264]: Next returned nil segment
Aug 18 10:12:29.579881340 (232h46m51.538978599s) [upload=242264]: Waiting for error group managing segments...
Aug 18 10:12:29.712502827 (232h46m51.671600067s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.715208093 (232h46m51.674305348s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.724656470 (232h46m51.683753719s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.731762008 (232h46m51.690859259s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.734744520 (232h46m51.693841759s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.735571428 (232h46m51.694668675s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.743776476 (232h46m51.702873735s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.755185797 (232h46m51.714283052s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.757620693 (232h46m51.716717925s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.761051617 (232h46m51.720148856s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.762461116 (232h46m51.721558363s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.779979392 (232h46m51.739076639s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.790560077 (232h46m51.749657337s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805160589 (232h46m51.764257838s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805207627 (232h46m51.764304874s) [seg_pos={0 0}, upload=242264]: Segment reached optimal threshold of 14 pieces.
Aug 18 10:12:29.805563963 (232h46m51.764661212s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WutpFSoqFPT4wPcAAsjsZD5WZy8boff1HmVefLHLyTfKNndy6): protocol: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection
Aug 18 10:12:29.805655820 (232h46m51.764753051s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection
Aug 18 10:12:29.805663766 (232h46m51.764761018s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Lo4tfpgtTeQbcNqfCb3zGHjEdRJUdf7hkThS9vfPgDvVLuEMW): protocol: expected piece hash; context canceled; EOF
Aug 18 10:12:29.805702123 (232h46m51.764799358s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gjesgEXSFr1aGyFgASrVRJgEHTne1KichuGqXwMjN1hHBt4kz): protocol: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection
Aug 18 10:12:29.805719533 (232h46m51.764816789s) [seg_pos={0 0}, upload=242264]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 18 10:12:29.805802398 (232h46m51.764899654s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 18 10:12:29.805809056 (232h46m51.764906311s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 18 10:12:29.954733853 (232h46m51.913831092s) [upload=242264]: Done uploading segments...
Aug 18 10:12:29.954776438 (232h46m51.913873679s) [upload=242264]: Upload finished. err: <nil>
Duration: 2.52180112s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [..............X...X.......X.....X...........................] duration:
915.021552ms
FAIL (2025-08-18 10:14:11 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 915.056506ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [...X............................X.........X...............X.] duration:
2.345538854s
FAIL (2025-08-18 10:13:42 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 18 10:12:28.814497391 (232h46m50.773594644s) [upload=242264]: Starting upload
Aug 18 10:12:28.814568850 (232h46m50.773666099s) [upload=242264]: Done starting upload
Aug 18 10:12:28.814652939 (232h46m50.773750187s) [upload=242264]: Uploading segments...
Aug 18 10:12:28.814925344 (232h46m50.77402259s) [upload=242264]: Got next segment. Inline: false
Aug 18 10:12:28.814972606 (232h46m50.774069851s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 18 10:12:28.814977665 (232h46m50.774074911s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 18 10:12:29.014812974 (232h46m50.973910208s) [seg_pos={0 0}, upload=242264]: Begin upload segment...
Aug 18 10:12:29.014870782 (232h46m50.973968016s) [seg_pos={0 0}, upload=242264]: Done begin upload segment.
Aug 18 10:12:29.014928044 (232h46m50.974025289s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.014949880 (232h46m50.974047113s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015381585 (232h46m50.974478838s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015505267 (232h46m50.974602528s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015688171 (232h46m50.974785417s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015899786 (232h46m50.97499704s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015982477 (232h46m50.975079722s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016282151 (232h46m50.975379399s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016313105 (232h46m50.975410357s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016584018 (232h46m50.975681263s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016879991 (232h46m50.975977246s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.017677212 (232h46m50.976774469s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018244920 (232h46m50.977342168s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018405683 (232h46m50.977502923s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018700697 (232h46m50.977797953s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018936705 (232h46m50.978033954s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019352075 (232h46m50.978449313s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019396481 (232h46m50.978493728s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.579861974 (232h46m51.538959224s) [upload=242264]: Next returned nil segment
Aug 18 10:12:29.579881340 (232h46m51.538978599s) [upload=242264]: Waiting for error group managing segments...
Aug 18 10:12:29.712502827 (232h46m51.671600067s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.715208093 (232h46m51.674305348s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.724656470 (232h46m51.683753719s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.731762008 (232h46m51.690859259s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.734744520 (232h46m51.693841759s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.735571428 (232h46m51.694668675s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.743776476 (232h46m51.702873735s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.755185797 (232h46m51.714283052s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.757620693 (232h46m51.716717925s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.761051617 (232h46m51.720148856s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.762461116 (232h46m51.721558363s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.779979392 (232h46m51.739076639s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.790560077 (232h46m51.749657337s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805160589 (232h46m51.764257838s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805207627 (232h46m51.764304874s) [seg_pos={0 0}, upload=242264]: Segment reached optimal threshold of 14 pieces.
Aug 18 10:12:29.805563963 (232h46m51.764661212s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WutpFSoqFPT4wPcAAsjsZD5WZy8boff1HmVefLHLyTfKNndy6): protocol: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection
Aug 18 10:12:29.805655820 (232h46m51.764753051s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection
Aug 18 10:12:29.805663766 (232h46m51.764761018s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Lo4tfpgtTeQbcNqfCb3zGHjEdRJUdf7hkThS9vfPgDvVLuEMW): protocol: expected piece hash; context canceled; EOF
Aug 18 10:12:29.805702123 (232h46m51.764799358s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gjesgEXSFr1aGyFgASrVRJgEHTne1KichuGqXwMjN1hHBt4kz): protocol: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection
Aug 18 10:12:29.805719533 (232h46m51.764816789s) [seg_pos={0 0}, upload=242264]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 18 10:12:29.805802398 (232h46m51.764899654s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 18 10:12:29.805809056 (232h46m51.764906311s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 18 10:12:29.954733853 (232h46m51.913831092s) [upload=242264]: Done uploading segments...
Aug 18 10:12:29.954776438 (232h46m51.913873679s) [upload=242264]: Upload finished. err: <nil>
Duration: 2.3456471s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [...............X...X.......X.....X..........................] duration:
1.055768725s
FAIL (2025-08-18 10:13:26 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.05579957s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [....X............................X.........X...............X] duration:
1.607318654s
FAIL (2025-08-18 10:12:30 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 18 10:12:28.814497391 (232h46m50.773594644s) [upload=242264]: Starting upload
Aug 18 10:12:28.814568850 (232h46m50.773666099s) [upload=242264]: Done starting upload
Aug 18 10:12:28.814652939 (232h46m50.773750187s) [upload=242264]: Uploading segments...
Aug 18 10:12:28.814925344 (232h46m50.77402259s) [upload=242264]: Got next segment. Inline: false
Aug 18 10:12:28.814972606 (232h46m50.774069851s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 18 10:12:28.814977665 (232h46m50.774074911s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 18 10:12:29.014812974 (232h46m50.973910208s) [seg_pos={0 0}, upload=242264]: Begin upload segment...
Aug 18 10:12:29.014870782 (232h46m50.973968016s) [seg_pos={0 0}, upload=242264]: Done begin upload segment.
Aug 18 10:12:29.014928044 (232h46m50.974025289s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.014949880 (232h46m50.974047113s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015381585 (232h46m50.974478838s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015505267 (232h46m50.974602528s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015688171 (232h46m50.974785417s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015899786 (232h46m50.97499704s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.015982477 (232h46m50.975079722s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016282151 (232h46m50.975379399s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016313105 (232h46m50.975410357s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016584018 (232h46m50.975681263s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.016879991 (232h46m50.975977246s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.017677212 (232h46m50.976774469s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018244920 (232h46m50.977342168s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018405683 (232h46m50.977502923s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018700697 (232h46m50.977797953s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.018936705 (232h46m50.978033954s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019352075 (232h46m50.978449313s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.019396481 (232h46m50.978493728s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Uploading piece...
Aug 18 10:12:29.579861974 (232h46m51.538959224s) [upload=242264]: Next returned nil segment
Aug 18 10:12:29.579881340 (232h46m51.538978599s) [upload=242264]: Waiting for error group managing segments...
Aug 18 10:12:29.712502827 (232h46m51.671600067s) [noise=true , address=45.91.116.10:4089 , piece_id=XOG3V4LVYPEMXH2VUY7EU6J2UPUXLQOS2QCOOLSENAKZCI6VATCA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.715208093 (232h46m51.674305348s) [noise=true , address=45.91.116.9:4276 , piece_id=BPZRONJ4QLYKXN2DXWT6HUJY46XX6HJTW4RXUARWENXTHKTV3RBQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.724656470 (232h46m51.683753719s) [noise=true , address=77.90.31.10:4546 , piece_id=AO3BIDSPLTDXFAY2W44EOTNIFPYIKA7JKVEHXPJCZDOVSMJUZQKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.731762008 (232h46m51.690859259s) [noise=true , address=77.90.31.17:4803 , piece_id=XQ3RBITYEJA7MIRWFJCLI5ER2I4GYVRMPTVL36EL6LE4AN3Q7YKA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.734744520 (232h46m51.693841759s) [noise=true , address=104.36.82.40:4948 , piece_id=NLZYWHCJFDSGXN2MIOCNSYYPYVYLOKK36XEJDPEBYC2ARAZOYFUQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.735571428 (232h46m51.694668675s) [noise=true , address=77.90.31.18:4820 , piece_id=QZPNXPC53ZYQORUVBHJUUTULHXVUTBQQE6ZIB5KMV3NWZOET555Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.743776476 (232h46m51.702873735s) [noise=true , address=104.36.82.44:4874 , piece_id=3OE2CNTBRKN2MNRGQWKKYFEUBB7G5NT7DAUTXSNKFZV4QHFKELMA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.755185797 (232h46m51.714283052s) [noise=true , address=104.36.82.5:4856 , piece_id=SADHJQCNH5WBL3BF72DBAXU7WX7PTC5VD3HJQOSFCSUBEQLEWOLQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.757620693 (232h46m51.716717925s) [noise=true , address=104.36.82.59:4310 , piece_id=XWJ4ZN5HQF3UWC33CW2S5V2LBVFBDUTCG5QPJR4Z2V4IFHTYEHRQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.761051617 (232h46m51.720148856s) [noise=true , address=77.90.31.21:4898 , piece_id=XDT5NWY4S4ZIHZSQB7Z22THTVDD6K4RQF6RL77VBKCE46GKB6TBA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.762461116 (232h46m51.721558363s) [noise=true , address=104.36.82.58:4269 , piece_id=MXGYA254SLQKG3ETVYB7DICLJMIGWLDXDKTVF2774ZJFVCKI2Q2Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.779979392 (232h46m51.739076639s) [noise=true , address=77.90.31.19:4859 , piece_id=ZVPCMYWL4L42JCIVFP74DLSTTPSMYAQZ4NMKSJOKSK4OKHHAWUHQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.790560077 (232h46m51.749657337s) [noise=true , address=104.36.82.23:4446 , piece_id=JG6D7WIT5CK24XJWTFZA3HYLXO3OX2BBFIK4KAD3KOB3D6ZEWQ4Q, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805160589 (232h46m51.764257838s) [noise=true , address=104.36.82.60:4343 , piece_id=P7M5MQRC2KMDX53YIDBOVW6OP5TTLKX6L4RR25VMQCBZEEJDHZYA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: <nil>
Aug 18 10:12:29.805207627 (232h46m51.764304874s) [seg_pos={0 0}, upload=242264]: Segment reached optimal threshold of 14 pieces.
Aug 18 10:12:29.805563963 (232h46m51.764661212s) [noise=true , address=79.127.201.215:4351 , piece_id=WGXO3I4TJRC7MBVCMBYYV3RTKKGUV5ZXXTE2EYD62D7MQCECIBUA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WutpFSoqFPT4wPcAAsjsZD5WZy8boff1HmVefLHLyTfKNndy6): protocol: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48386->79.127.201.215:4351: use of closed network connection
Aug 18 10:12:29.805655820 (232h46m51.764753051s) [noise=true , address=66.23.193.175:4554 , piece_id=YLWT2SV7TG3Q5WLNVTRUOBU2HJODI2N6R25W7YMOXWCA5G6GF6SA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38426->66.23.193.175:4554: use of closed network connection
Aug 18 10:12:29.805663766 (232h46m51.764761018s) [noise=true , address=79.127.219.48:4103 , piece_id=ABAN3DMSZH3RELNXG4NDGMI5W22GP634WKU3HN3MOZ34NBLPORFQ, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Lo4tfpgtTeQbcNqfCb3zGHjEdRJUdf7hkThS9vfPgDvVLuEMW): protocol: expected piece hash; context canceled; EOF
Aug 18 10:12:29.805702123 (232h46m51.764799358s) [noise=true , address=77.90.31.16:4757 , piece_id=Y4XVMAKD54BXT3SFIG7NYOZXRRO5QOXZKUQMGMHZMNVGFZNAN5NA, seg_pos={0 0}, upload=242264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gjesgEXSFr1aGyFgASrVRJgEHTne1KichuGqXwMjN1hHBt4kz): protocol: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50918->77.90.31.16:4757: use of closed network connection
Aug 18 10:12:29.805719533 (232h46m51.764816789s) [seg_pos={0 0}, upload=242264]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 18 10:12:29.805802398 (232h46m51.764899654s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 18 10:12:29.805809056 (232h46m51.764906311s) [upload=242264]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 18 10:12:29.954733853 (232h46m51.913831092s) [upload=242264]: Done uploading segments...
Aug 18 10:12:29.954776438 (232h46m51.913873679s) [upload=242264]: Upload finished. err: <nil>
Duration: 1.607372171s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [................X...X.......X.....X.........................] duration:
866.069064ms
FAIL (2025-08-18 10:12:23 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 866.198102ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [.....X............................X.........X...............] duration:
2.447653654s
FAIL (2025-08-18 10:12:10 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 10:03:41.752795958 (232h38m3.711893203s) [upload=242111]: Starting upload Aug 18 10:03:41.752851974 (232h38m3.711949228s) [upload=242111]: Done starting upload Aug 18 10:03:41.753075790 (232h38m3.712173024s) [upload=242111]: Uploading segments... Aug 18 10:03:41.753250644 (232h38m3.712347889s) [upload=242111]: Got next segment. Inline: false Aug 18 10:03:41.753287258 (232h38m3.712384506s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 10:03:41.753292103 (232h38m3.712389348s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 10:03:41.843788201 (232h38m3.80288545s) [upload=242111]: Next segment err: metaclient: Too Many Requests Aug 18 10:03:41.843809998 (232h38m3.802907243s) [upload=242111]: Done uploading segments... Aug 18 10:03:41.843835660 (232h38m3.802932906s) [upload=242111]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.447775138s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [......X............................X.........X..............] duration:
2.317386619s
FAIL (2025-08-18 10:11:54 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 10:03:41.752795958 (232h38m3.711893203s) [upload=242111]: Starting upload Aug 18 10:03:41.752851974 (232h38m3.711949228s) [upload=242111]: Done starting upload Aug 18 10:03:41.753075790 (232h38m3.712173024s) [upload=242111]: Uploading segments... Aug 18 10:03:41.753250644 (232h38m3.712347889s) [upload=242111]: Got next segment. Inline: false Aug 18 10:03:41.753287258 (232h38m3.712384506s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 10:03:41.753292103 (232h38m3.712389348s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 10:03:41.843788201 (232h38m3.80288545s) [upload=242111]: Next segment err: metaclient: Too Many Requests Aug 18 10:03:41.843809998 (232h38m3.802907243s) [upload=242111]: Done uploading segments... Aug 18 10:03:41.843835660 (232h38m3.802932906s) [upload=242111]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.317433035s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.................X...X.......X.....X........................] duration:
805.233966ms
FAIL (2025-08-18 10:11:42 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 805.26804ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [.......X............................X.........X.............] duration:
2.327066316s
FAIL (2025-08-18 10:11:09 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 10:03:41.752795958 (232h38m3.711893203s) [upload=242111]: Starting upload Aug 18 10:03:41.752851974 (232h38m3.711949228s) [upload=242111]: Done starting upload Aug 18 10:03:41.753075790 (232h38m3.712173024s) [upload=242111]: Uploading segments... Aug 18 10:03:41.753250644 (232h38m3.712347889s) [upload=242111]: Got next segment. Inline: false Aug 18 10:03:41.753287258 (232h38m3.712384506s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 10:03:41.753292103 (232h38m3.712389348s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 10:03:41.843788201 (232h38m3.80288545s) [upload=242111]: Next segment err: metaclient: Too Many Requests Aug 18 10:03:41.843809998 (232h38m3.802907243s) [upload=242111]: Done uploading segments... Aug 18 10:03:41.843835660 (232h38m3.802932906s) [upload=242111]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.327096948s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [..................X...X.......X.....X.......................] duration:
831.979739ms
FAIL (2025-08-18 10:10:43 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 832.026659ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [........X............................X.........X............] duration:
2.386386384s
FAIL (2025-08-18 10:10:39 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 10:03:41.752795958 (232h38m3.711893203s) [upload=242111]: Starting upload Aug 18 10:03:41.752851974 (232h38m3.711949228s) [upload=242111]: Done starting upload Aug 18 10:03:41.753075790 (232h38m3.712173024s) [upload=242111]: Uploading segments... Aug 18 10:03:41.753250644 (232h38m3.712347889s) [upload=242111]: Got next segment. Inline: false Aug 18 10:03:41.753287258 (232h38m3.712384506s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 10:03:41.753292103 (232h38m3.712389348s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 10:03:41.843788201 (232h38m3.80288545s) [upload=242111]: Next segment err: metaclient: Too Many Requests Aug 18 10:03:41.843809998 (232h38m3.802907243s) [upload=242111]: Done uploading segments... Aug 18 10:03:41.843835660 (232h38m3.802932906s) [upload=242111]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.386416085s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [.........X............................X.........X...........] duration:
2.47188124s
FAIL (2025-08-18 10:10:14 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 10:03:41.752795958 (232h38m3.711893203s) [upload=242111]: Starting upload Aug 18 10:03:41.752851974 (232h38m3.711949228s) [upload=242111]: Done starting upload Aug 18 10:03:41.753075790 (232h38m3.712173024s) [upload=242111]: Uploading segments... Aug 18 10:03:41.753250644 (232h38m3.712347889s) [upload=242111]: Got next segment. Inline: false Aug 18 10:03:41.753287258 (232h38m3.712384506s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 10:03:41.753292103 (232h38m3.712389348s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 10:03:41.843788201 (232h38m3.80288545s) [upload=242111]: Next segment err: metaclient: Too Many Requests Aug 18 10:03:41.843809998 (232h38m3.802907243s) [upload=242111]: Done uploading segments... Aug 18 10:03:41.843835660 (232h38m3.802932906s) [upload=242111]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.471931865s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [..........X............................X.........X..........] duration:
2.631360707s
FAIL (2025-08-18 10:09:57 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 10:03:41.752795958 (232h38m3.711893203s) [upload=242111]: Starting upload Aug 18 10:03:41.752851974 (232h38m3.711949228s) [upload=242111]: Done starting upload Aug 18 10:03:41.753075790 (232h38m3.712173024s) [upload=242111]: Uploading segments... Aug 18 10:03:41.753250644 (232h38m3.712347889s) [upload=242111]: Got next segment. Inline: false Aug 18 10:03:41.753287258 (232h38m3.712384506s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 10:03:41.753292103 (232h38m3.712389348s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 10:03:41.843788201 (232h38m3.80288545s) [upload=242111]: Next segment err: metaclient: Too Many Requests Aug 18 10:03:41.843809998 (232h38m3.802907243s) [upload=242111]: Done uploading segments... Aug 18 10:03:41.843835660 (232h38m3.802932906s) [upload=242111]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.631399345s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [...................X...X.......X.....X......................] duration:
2.126110008s
FAIL (2025-08-18 10:09:55 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.126143485s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [...........X............................X.........X.........] duration:
2.363326355s
FAIL (2025-08-18 10:09:24 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 10:03:41.752795958 (232h38m3.711893203s) [upload=242111]: Starting upload Aug 18 10:03:41.752851974 (232h38m3.711949228s) [upload=242111]: Done starting upload Aug 18 10:03:41.753075790 (232h38m3.712173024s) [upload=242111]: Uploading segments... Aug 18 10:03:41.753250644 (232h38m3.712347889s) [upload=242111]: Got next segment. Inline: false Aug 18 10:03:41.753287258 (232h38m3.712384506s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 10:03:41.753292103 (232h38m3.712389348s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 10:03:41.843788201 (232h38m3.80288545s) [upload=242111]: Next segment err: metaclient: Too Many Requests Aug 18 10:03:41.843809998 (232h38m3.802907243s) [upload=242111]: Done uploading segments... Aug 18 10:03:41.843835660 (232h38m3.802932906s) [upload=242111]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.363362291s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [............X............................X.........X........] duration:
2.372296188s
FAIL (2025-08-18 10:08:50 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 10:03:41.752795958 (232h38m3.711893203s) [upload=242111]: Starting upload Aug 18 10:03:41.752851974 (232h38m3.711949228s) [upload=242111]: Done starting upload Aug 18 10:03:41.753075790 (232h38m3.712173024s) [upload=242111]: Uploading segments... Aug 18 10:03:41.753250644 (232h38m3.712347889s) [upload=242111]: Got next segment. Inline: false Aug 18 10:03:41.753287258 (232h38m3.712384506s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 10:03:41.753292103 (232h38m3.712389348s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 10:03:41.843788201 (232h38m3.80288545s) [upload=242111]: Next segment err: metaclient: Too Many Requests Aug 18 10:03:41.843809998 (232h38m3.802907243s) [upload=242111]: Done uploading segments... Aug 18 10:03:41.843835660 (232h38m3.802932906s) [upload=242111]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.372326247s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [....................X...X.......X.....X.....................] duration:
845.012571ms
FAIL (2025-08-18 10:08:49 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 845.11897ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [.............X............................X.........X.......] duration:
2.43114819s
FAIL (2025-08-18 10:08:22 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 10:03:41.752795958 (232h38m3.711893203s) [upload=242111]: Starting upload Aug 18 10:03:41.752851974 (232h38m3.711949228s) [upload=242111]: Done starting upload Aug 18 10:03:41.753075790 (232h38m3.712173024s) [upload=242111]: Uploading segments... Aug 18 10:03:41.753250644 (232h38m3.712347889s) [upload=242111]: Got next segment. Inline: false Aug 18 10:03:41.753287258 (232h38m3.712384506s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 10:03:41.753292103 (232h38m3.712389348s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 10:03:41.843788201 (232h38m3.80288545s) [upload=242111]: Next segment err: metaclient: Too Many Requests Aug 18 10:03:41.843809998 (232h38m3.802907243s) [upload=242111]: Done uploading segments... Aug 18 10:03:41.843835660 (232h38m3.802932906s) [upload=242111]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.431182077s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.....................X...X.......X.....X....................] duration:
848.42371ms
FAIL (2025-08-18 10:08:02 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 848.46219ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [..............X............................X.........X......] duration:
2.649234979s
FAIL (2025-08-18 10:07:35 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 10:03:41.752795958 (232h38m3.711893203s) [upload=242111]: Starting upload Aug 18 10:03:41.752851974 (232h38m3.711949228s) [upload=242111]: Done starting upload Aug 18 10:03:41.753075790 (232h38m3.712173024s) [upload=242111]: Uploading segments... Aug 18 10:03:41.753250644 (232h38m3.712347889s) [upload=242111]: Got next segment. Inline: false Aug 18 10:03:41.753287258 (232h38m3.712384506s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 10:03:41.753292103 (232h38m3.712389348s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 10:03:41.843788201 (232h38m3.80288545s) [upload=242111]: Next segment err: metaclient: Too Many Requests Aug 18 10:03:41.843809998 (232h38m3.802907243s) [upload=242111]: Done uploading segments... Aug 18 10:03:41.843835660 (232h38m3.802932906s) [upload=242111]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.649289782s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [...............X............................X.........X.....] duration:
2.394155228s
FAIL (2025-08-18 10:07:15 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 10:03:41.752795958 (232h38m3.711893203s) [upload=242111]: Starting upload Aug 18 10:03:41.752851974 (232h38m3.711949228s) [upload=242111]: Done starting upload Aug 18 10:03:41.753075790 (232h38m3.712173024s) [upload=242111]: Uploading segments... Aug 18 10:03:41.753250644 (232h38m3.712347889s) [upload=242111]: Got next segment. Inline: false Aug 18 10:03:41.753287258 (232h38m3.712384506s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 10:03:41.753292103 (232h38m3.712389348s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 10:03:41.843788201 (232h38m3.80288545s) [upload=242111]: Next segment err: metaclient: Too Many Requests Aug 18 10:03:41.843809998 (232h38m3.802907243s) [upload=242111]: Done uploading segments... Aug 18 10:03:41.843835660 (232h38m3.802932906s) [upload=242111]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.394186028s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [......................X...X.......X.....X...................] duration:
821.798161ms
FAIL (2025-08-18 10:07:12 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 821.836061ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [................X............................X.........X....] duration:
2.468110445s
FAIL (2025-08-18 10:06:28 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 10:03:41.752795958 (232h38m3.711893203s) [upload=242111]: Starting upload Aug 18 10:03:41.752851974 (232h38m3.711949228s) [upload=242111]: Done starting upload Aug 18 10:03:41.753075790 (232h38m3.712173024s) [upload=242111]: Uploading segments... Aug 18 10:03:41.753250644 (232h38m3.712347889s) [upload=242111]: Got next segment. Inline: false Aug 18 10:03:41.753287258 (232h38m3.712384506s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 10:03:41.753292103 (232h38m3.712389348s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 10:03:41.843788201 (232h38m3.80288545s) [upload=242111]: Next segment err: metaclient: Too Many Requests Aug 18 10:03:41.843809998 (232h38m3.802907243s) [upload=242111]: Done uploading segments... Aug 18 10:03:41.843835660 (232h38m3.802932906s) [upload=242111]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.468146913s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.......................X...X.......X.....X..................] duration:
834.137673ms
FAIL (2025-08-18 10:06:18 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 09:52:23.912123544 (232h26m45.871220798s) [upload=241916]: Starting upload Aug 18 09:52:23.912211839 (232h26m45.871309081s) [upload=241916]: Done starting upload Aug 18 09:52:23.912317119 (232h26m45.871414351s) [upload=241916]: Uploading segments... Aug 18 09:52:23.912423931 (232h26m45.871521182s) [upload=241916]: Got next segment. Inline: true Aug 18 09:52:23.912433457 (232h26m45.871530708s) [upload=241916]: Waiting for error group managing segments... Aug 18 09:52:23.912510827 (232h26m45.871608073s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 09:52:23.912517353 (232h26m45.8716146s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 09:52:23.912521250 (232h26m45.871618497s) [upload=241916]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 09:52:24.002411251 (232h26m45.961508537s) [upload=241916]: Done uploading segments... Aug 18 09:52:24.002436032 (232h26m45.961533294s) [upload=241916]: Upload finished. err: metaclient: Too Many Requests
Duration: 834.168533ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [.................X............................X.........X...] duration:
3.939528094s
FAIL (2025-08-18 10:05:58 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 10:03:41.752795958 (232h38m3.711893203s) [upload=242111]: Starting upload Aug 18 10:03:41.752851974 (232h38m3.711949228s) [upload=242111]: Done starting upload Aug 18 10:03:41.753075790 (232h38m3.712173024s) [upload=242111]: Uploading segments... Aug 18 10:03:41.753250644 (232h38m3.712347889s) [upload=242111]: Got next segment. Inline: false Aug 18 10:03:41.753287258 (232h38m3.712384506s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 10:03:41.753292103 (232h38m3.712389348s) [upload=242111]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 10:03:41.843788201 (232h38m3.80288545s) [upload=242111]: Next segment err: metaclient: Too Many Requests Aug 18 10:03:41.843809998 (232h38m3.802907243s) [upload=242111]: Done uploading segments... Aug 18 10:03:41.843835660 (232h38m3.802932906s) [upload=242111]: Upload finished. err: metaclient: Too Many Requests
Duration: 3.939576325s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|