Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [........X..X........X.......................................] duration:
2.570840383s
FAIL (2025-08-19 05:12:14 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.570871441s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [.....X..XX.......................X.......X..................] duration:
829.903646ms
FAIL (2025-08-19 05:11:36 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 829.944209ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [.........X..X........X......................................] duration:
2.485809914s
FAIL (2025-08-19 05:11:28 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.485849338s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [......X..XX.......................X.......X.................] duration:
820.54969ms
FAIL (2025-08-19 05:11:13 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 820.57969ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [.......X..XX.......................X.......X................] duration:
820.362416ms
FAIL (2025-08-19 05:10:44 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 820.409334ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [..........X..X........X.....................................] duration:
2.392388905s
FAIL (2025-08-19 05:10:18 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.3924817s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [........X..XX.......................X.......X...............] duration:
819.744646ms
FAIL (2025-08-19 05:10:04 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 819.785ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [.........X..XX.......................X.......X..............] duration:
829.232612ms
FAIL (2025-08-19 05:09:43 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 829.303549ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [...........X..X........X....................................] duration:
2.429454521s
FAIL (2025-08-19 05:09:32 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.429494509s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [............X..X........X...................................] duration:
2.407365753s
FAIL (2025-08-19 05:08:54 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.407400191s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [..........X..XX.......................X.......X.............] duration:
824.288913ms
FAIL (2025-08-19 05:08:39 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 824.319714ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [...........X..XX.......................X.......X............] duration:
810.467296ms
FAIL (2025-08-19 05:08:06 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 810.500587ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [.............X..X........X..................................] duration:
2.280988497s
FAIL (2025-08-19 05:07:45 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.281018964s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [............X..XX.......................X.......X...........] duration:
818.246897ms
FAIL (2025-08-19 05:07:29 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 818.290777ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [.............X..XX.......................X.......X..........] duration:
814.461016ms
FAIL (2025-08-19 05:07:08 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 814.492994ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [..............X..X........X.................................] duration:
2.412341065s
FAIL (2025-08-19 05:06:44 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.412374016s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [..............X..XX.......................X.......X.........] duration:
814.955451ms
FAIL (2025-08-19 05:06:21 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 814.986561ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [...............X..X........X................................] duration:
2.800317193s
FAIL (2025-08-19 05:05:52 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.800350939s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [...............X..XX.......................X.......X........] duration:
844.088996ms
FAIL (2025-08-19 05:05:50 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 844.177551ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [X...............X..X........X...............................] duration:
3.012786461s
FAIL (2025-08-19 05:05:01 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 3.012821009s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [................X..XX.......................X.......X.......] duration:
812.080826ms
FAIL (2025-08-19 05:04:47 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 812.11135ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [.X...............X..X........X..............................] duration:
2.373963471s
FAIL (2025-08-19 05:04:39 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.373995047s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [..X...............X..X........X.............................] duration:
2.522330898s
FAIL (2025-08-19 05:04:18 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.522414122s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [.................X..XX.......................X.......X......] duration:
983.750967ms
FAIL (2025-08-19 05:03:52 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 983.78171ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [...X...............X..X........X............................] duration:
2.458796038s
FAIL (2025-08-19 05:03:33 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.458829927s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [..................X..XX.......................X.......X.....] duration:
1.088874904s
FAIL (2025-08-19 05:03:31 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.08895927s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [...................X..XX.......................X.......X....] duration:
815.284284ms
FAIL (2025-08-19 05:02:52 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 815.31294ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [....X...............X..X........X...........................] duration:
2.391739236s
FAIL (2025-08-19 05:02:39 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.391788149s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [.....X...............X..X........X..........................] duration:
2.37287803s
FAIL (2025-08-19 05:02:16 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.372981824s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [....................X..XX.......................X.......X...] duration:
840.503696ms
FAIL (2025-08-19 05:02:08 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 840.538503ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [X.....X...............X..X........X.........................] duration:
3.291918004s
FAIL (2025-08-19 05:02:00 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 3.291953555s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [.....................X..XX.......................X.......X..] duration:
897.235226ms
FAIL (2025-08-19 05:01:22 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 897.27254ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.X.....X...............X..X........X........................] duration:
2.36590101s
FAIL (2025-08-19 05:00:49 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.366001793s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [......................X..XX.......................X.......X.] duration:
844.252264ms
FAIL (2025-08-19 05:00:39 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 844.286818ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [..X.....X...............X..X........X.......................] duration:
2.655201457s
FAIL (2025-08-19 05:00:11 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.65530177s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [.......................X..XX.......................X.......X] duration:
412.390965ms
FAIL (2025-08-19 04:59:45 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 412.416021ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [...X.....X...............X..X........X......................] duration:
2.293431021s
FAIL (2025-08-19 04:59:25 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.293585899s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [....X.....X...............X..X........X.....................] duration:
2.323502396s
FAIL (2025-08-19 04:59:09 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.323530028s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [........................X..XX.......................X.......] duration:
837.581433ms
FAIL (2025-08-19 04:58:33 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:52:45.130391488 (251h27m7.089488733s) [upload=261727]: Starting upload Aug 19 04:52:45.130454864 (251h27m7.089552112s) [upload=261727]: Done starting upload Aug 19 04:52:45.130534073 (251h27m7.089631311s) [upload=261727]: Uploading segments... Aug 19 04:52:45.130611587 (251h27m7.089708842s) [upload=261727]: Got next segment. Inline: true Aug 19 04:52:45.130618784 (251h27m7.089716041s) [upload=261727]: Waiting for error group managing segments... Aug 19 04:52:45.130684700 (251h27m7.089781945s) [upload=261727]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:52:45.130690009 (251h27m7.089787255s) [upload=261727]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:52:45.130693844 (251h27m7.08979109s) [upload=261727]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:52:45.269396988 (251h27m7.228494236s) [upload=261727]: Done uploading segments... Aug 19 04:52:45.269420151 (251h27m7.228517405s) [upload=261727]: Upload finished. err: metaclient: Too Many Requests
Duration: 837.621456ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.....X.....X...............X..X........X....................] duration:
2.571335844s
FAIL (2025-08-19 04:58:11 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.571365402s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.........................X..XX.......................X......] duration:
809.05322ms
FAIL (2025-08-19 04:57:34 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:52:45.130391488 (251h27m7.089488733s) [upload=261727]: Starting upload Aug 19 04:52:45.130454864 (251h27m7.089552112s) [upload=261727]: Done starting upload Aug 19 04:52:45.130534073 (251h27m7.089631311s) [upload=261727]: Uploading segments... Aug 19 04:52:45.130611587 (251h27m7.089708842s) [upload=261727]: Got next segment. Inline: true Aug 19 04:52:45.130618784 (251h27m7.089716041s) [upload=261727]: Waiting for error group managing segments... Aug 19 04:52:45.130684700 (251h27m7.089781945s) [upload=261727]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:52:45.130690009 (251h27m7.089787255s) [upload=261727]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:52:45.130693844 (251h27m7.08979109s) [upload=261727]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:52:45.269396988 (251h27m7.228494236s) [upload=261727]: Done uploading segments... Aug 19 04:52:45.269420151 (251h27m7.228517405s) [upload=261727]: Upload finished. err: metaclient: Too Many Requests
Duration: 809.090027ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [......X.....X...............X..X........X...................] duration:
2.380415693s
FAIL (2025-08-19 04:57:23 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.380478974s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [..........................X..XX.......................X.....] duration:
928.102049ms
FAIL (2025-08-19 04:57:06 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:52:45.130391488 (251h27m7.089488733s) [upload=261727]: Starting upload Aug 19 04:52:45.130454864 (251h27m7.089552112s) [upload=261727]: Done starting upload Aug 19 04:52:45.130534073 (251h27m7.089631311s) [upload=261727]: Uploading segments... Aug 19 04:52:45.130611587 (251h27m7.089708842s) [upload=261727]: Got next segment. Inline: true Aug 19 04:52:45.130618784 (251h27m7.089716041s) [upload=261727]: Waiting for error group managing segments... Aug 19 04:52:45.130684700 (251h27m7.089781945s) [upload=261727]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:52:45.130690009 (251h27m7.089787255s) [upload=261727]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:52:45.130693844 (251h27m7.08979109s) [upload=261727]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:52:45.269396988 (251h27m7.228494236s) [upload=261727]: Done uploading segments... Aug 19 04:52:45.269420151 (251h27m7.228517405s) [upload=261727]: Upload finished. err: metaclient: Too Many Requests
Duration: 928.135728ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.......X.....X...............X..X........X..................] duration:
2.467102936s
FAIL (2025-08-19 04:56:26 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.46713506s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [...........................X..XX.......................X....] duration:
1.017585369s
FAIL (2025-08-19 04:56:21 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:52:45.130391488 (251h27m7.089488733s) [upload=261727]: Starting upload Aug 19 04:52:45.130454864 (251h27m7.089552112s) [upload=261727]: Done starting upload Aug 19 04:52:45.130534073 (251h27m7.089631311s) [upload=261727]: Uploading segments... Aug 19 04:52:45.130611587 (251h27m7.089708842s) [upload=261727]: Got next segment. Inline: true Aug 19 04:52:45.130618784 (251h27m7.089716041s) [upload=261727]: Waiting for error group managing segments... Aug 19 04:52:45.130684700 (251h27m7.089781945s) [upload=261727]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:52:45.130690009 (251h27m7.089787255s) [upload=261727]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:52:45.130693844 (251h27m7.08979109s) [upload=261727]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:52:45.269396988 (251h27m7.228494236s) [upload=261727]: Done uploading segments... Aug 19 04:52:45.269420151 (251h27m7.228517405s) [upload=261727]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.017620823s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [........X.....X...............X..X........X.................] duration:
2.527897552s
FAIL (2025-08-19 04:55:36 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.527932184s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [............................X..XX.......................X...] duration:
827.710132ms
FAIL (2025-08-19 04:55:17 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:52:45.130391488 (251h27m7.089488733s) [upload=261727]: Starting upload Aug 19 04:52:45.130454864 (251h27m7.089552112s) [upload=261727]: Done starting upload Aug 19 04:52:45.130534073 (251h27m7.089631311s) [upload=261727]: Uploading segments... Aug 19 04:52:45.130611587 (251h27m7.089708842s) [upload=261727]: Got next segment. Inline: true Aug 19 04:52:45.130618784 (251h27m7.089716041s) [upload=261727]: Waiting for error group managing segments... Aug 19 04:52:45.130684700 (251h27m7.089781945s) [upload=261727]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:52:45.130690009 (251h27m7.089787255s) [upload=261727]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:52:45.130693844 (251h27m7.08979109s) [upload=261727]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:52:45.269396988 (251h27m7.228494236s) [upload=261727]: Done uploading segments... Aug 19 04:52:45.269420151 (251h27m7.228517405s) [upload=261727]: Upload finished. err: metaclient: Too Many Requests
Duration: 827.738981ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.............................X..XX.......................X..] duration:
1.650023462s
FAIL (2025-08-19 04:54:32 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:52:45.130391488 (251h27m7.089488733s) [upload=261727]: Starting upload Aug 19 04:52:45.130454864 (251h27m7.089552112s) [upload=261727]: Done starting upload Aug 19 04:52:45.130534073 (251h27m7.089631311s) [upload=261727]: Uploading segments... Aug 19 04:52:45.130611587 (251h27m7.089708842s) [upload=261727]: Got next segment. Inline: true Aug 19 04:52:45.130618784 (251h27m7.089716041s) [upload=261727]: Waiting for error group managing segments... Aug 19 04:52:45.130684700 (251h27m7.089781945s) [upload=261727]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:52:45.130690009 (251h27m7.089787255s) [upload=261727]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:52:45.130693844 (251h27m7.08979109s) [upload=261727]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:52:45.269396988 (251h27m7.228494236s) [upload=261727]: Done uploading segments... Aug 19 04:52:45.269420151 (251h27m7.228517405s) [upload=261727]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.650051439s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.........X.....X...............X..X........X................] duration:
2.411261935s
FAIL (2025-08-19 04:54:22 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.411296881s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [..........X.....X...............X..X........X...............] duration:
2.929132814s
FAIL (2025-08-19 04:53:39 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.92917768s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|