Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [....X.......X...............................................] duration:
1.036296433s
FAIL (2025-08-19 05:31:45 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.036338179s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1.652246484s
FAIL (2025-08-19 05:31:32 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 05:31:31.048656329 (252h5m53.00775359s) [upload=262415]: Starting upload
Aug 19 05:31:31.048763241 (252h5m53.007860474s) [upload=262415]: Done starting upload
Aug 19 05:31:31.048858488 (252h5m53.007955734s) [upload=262415]: Uploading segments...
Aug 19 05:31:31.049824275 (252h5m53.008921521s) [upload=262415]: Got next segment. Inline: false
Aug 19 05:31:31.049880956 (252h5m53.008978192s) [upload=262415]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 05:31:31.049886788 (252h5m53.008984034s) [upload=262415]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 05:31:31.293571416 (252h5m53.252668673s) [seg_pos={0 0}, upload=262415]: Begin upload segment...
Aug 19 05:31:31.293641361 (252h5m53.252738607s) [seg_pos={0 0}, upload=262415]: Done begin upload segment.
Aug 19 05:31:31.293669212 (252h5m53.252766463s) [noise=true , address=104.36.82.36:4636 , piece_id=DLGWJ6FAW36ZI4BFR5WFOV2RHXQDQSRFTQLVY64HCQYEZ6232O6A, seg_pos={0 0}, upload=262415]: Uploading piece...
Aug 19 05:31:31.293693548 (252h5m53.252790821s) [noise=true , address=79.127.205.245:4236 , piece_id=JMRUFAAZGXSLHLYVFUYLPPNJUOIK4PKQZR2DL3B5VCRP6GWBB5EA, seg_pos={0 0}, upload=262415]: Uploading piece...
Aug 19 05:31:31.294075000 (252h5m53.253172248s) [noise=true , address=45.91.116.11:4137 , piece_id=4HXHLAMTTN7L2Y7OX4SKNRGI7FHAPNJVY6RJ6EFC7Z5CW67UPU7A, seg_pos={0 0}, upload=262415]: Uploading piece...
Aug 19 05:31:31.294382331 (252h5m53.253479578s) [noise=true , address=45.91.116.13:4169 , piece_id=25FBJW33TYVHTJHEYGOSHD3N2GYGW7PDNMIMGFSLBKRF7LVL7GFQ, seg_pos={0 0}, upload=262415]: Uploading piece...
Aug 19 05:31:31.294686892 (252h5m53.253784137s) [noise=true , address=104.36.82.23:4446 , piece_id=GRZIXHV6UDYWCBSIPDTAAEGFN6VLR2T22B7XKFDQJE4WSHELLXPQ, seg_pos={0 0}, upload=262415]: Uploading piece...
Aug 19 05:31:31.294837081 (252h5m53.253934335s) [noise=true , address=104.36.82.44:4895 , piece_id=TQHQYFVSIHONZNO6QBZORM3BBI4EJSQ75WF3BMTHFZAJJDKMRA7Q, seg_pos={0 0}, upload=262415]: Uploading piece...
Aug 19 05:31:31.294985989 (252h5m53.254083224s) [noise=true , address=45.91.116.12:4256 , piece_id=LSI63BLNFPZNM6AAMLQPAH4HEB25UV7UK4WY6BVTG3YP777ZRE5A, seg_pos={0 0}, upload=262415]: Uploading piece...
Aug 19 05:31:31.295359874 (252h5m53.254457122s) [noise=true , address=79.127.219.47:4062 , piece_id=ALSAEQIIBVCA7T76QIBVQ7JXMDGP74RPNMAFLDSLDPFDBS6LNICQ, seg_pos={0 0}, upload=262415]: Uploading piece...
Aug 19 05:31:31.295619926 (252h5m53.254717173s) [noise=true , address=77.90.31.12:4621 , piece_id=NFUPTPMDUDBPT2QEBK42EB5NOVE73FYOYUIVNGIYS6ODRPUUFYWA, seg_pos={0 0}, upload=262415]: Uploading piece...
Aug 19 05:31:31.296031735 (252h5m53.25512899s) [noise=true , address=77.90.31.21:4890 , piece_id=4WLT7FEV5Q65YHAIBASU5G4POBW5QRFFQ4H3PHSBL6HR73ZH4XQA, seg_pos={0 0}, upload=262415]: Uploading piece...
Aug 19 05:31:31.296399009 (252h5m53.255496263s) [noise=true , address=77.90.31.11:4573 , piece_id=RD5MDTHZMTDOAWN7E7K3FHLGG54XJI475WPC2LUZEH4MTUVP25SA, seg_pos={0 0}, upload=262415]: Uploading piece...
Aug 19 05:31:31.296552158 (252h5m53.255649409s) [noise=true , address=77.90.31.17:4800 , piece_id=OHYVHWG6RXCTIC2EE3B6OLJYGVU775GODZE2GUNWXSZHVYGYDGZA, seg_pos={0 0}, upload=262415]: Uploading piece...
Aug 19 05:31:31.296981842 (252h5m53.25607909s) [noise=true , address=79.127.219.38:4033 , piece_id=5MTCGC3URKN4MEQ7MKLMIRAOXPG3XIFOHFMTQENO46WTMJSCDVSA, seg_pos={0 0}, upload=262415]: Uploading piece...
Aug 19 05:31:31.297292084 (252h5m53.25638934s) [noise=true , address=104.36.82.60:4342 , piece_id=CUUNPNCIKAVK6DYWN2D3WQNA2H7R2YOYBE7HMC56RMIBNRQY6TZA, seg_pos={0 0}, upload=262415]: Uploading piece...
Aug 19 05:31:31.297602545 (252h5m53.256699801s) [noise=true , address=104.36.82.34:4563 , piece_id=QZE2LRF4FOODIJXF7YAGKOBNMVPZYXKPVRADHXIFTGELF75BJIQA, seg_pos={0 0}, upload=262415]: Uploading piece...
Aug 19 05:31:31.298085537 (252h5m53.257182796s) [noise=true , address=45.91.116.4:4941 , piece_id=XLHBRNY3ZMQRAJWGNRPDAA4AXTUEY4YQKLUMTLYWMDCZTGWMEQ6Q, seg_pos={0 0}, upload=262415]: Uploading piece...
Aug 19 05:31:31.298386390 (252h5m53.257483625s) [noise=true , address=77.90.31.14:4664 , piece_id=UBBQSNZWZX7ZYDXUI7QZ2LJWFPXPZZ6EZGDJKBI5ATGUFMN5IZAA, seg_pos={0 0}, upload=262415]: Uploading piece...
Aug 19 05:31:31.298559104 (252h5m53.257656361s) [noise=true , address=77.90.31.13:4636 , piece_id=3DXYBNJMHLR53B24BCWC43WYGSV7ATFEQRYJM2WCEFORPRNUMY4Q, seg_pos={0 0}, upload=262415]: Uploading piece...
Aug 19 05:31:31.858916737 (252h5m53.81801397s) [upload=262415]: Next returned nil segment
Aug 19 05:31:31.858937593 (252h5m53.818034848s) [upload=262415]: Waiting for error group managing segments...
Aug 19 05:31:32.003151706 (252h5m53.962248952s) [noise=true , address=77.90.31.17:4800 , piece_id=OHYVHWG6RXCTIC2EE3B6OLJYGVU775GODZE2GUNWXSZHVYGYDGZA, seg_pos={0 0}, upload=262415]: Done uploading piece. err: <nil>
Aug 19 05:31:32.004306627 (252h5m53.963403881s) [noise=true , address=77.90.31.14:4664 , piece_id=UBBQSNZWZX7ZYDXUI7QZ2LJWFPXPZZ6EZGDJKBI5ATGUFMN5IZAA, seg_pos={0 0}, upload=262415]: Done uploading piece. err: <nil>
Aug 19 05:31:32.004492468 (252h5m53.963589721s) [noise=true , address=45.91.116.11:4137 , piece_id=4HXHLAMTTN7L2Y7OX4SKNRGI7FHAPNJVY6RJ6EFC7Z5CW67UPU7A, seg_pos={0 0}, upload=262415]: Done uploading piece. err: <nil>
Aug 19 05:31:32.010462458 (252h5m53.969559696s) [noise=true , address=45.91.116.13:4169 , piece_id=25FBJW33TYVHTJHEYGOSHD3N2GYGW7PDNMIMGFSLBKRF7LVL7GFQ, seg_pos={0 0}, upload=262415]: Done uploading piece. err: <nil>
Aug 19 05:31:32.013518701 (252h5m53.972615937s) [noise=true , address=104.36.82.36:4636 , piece_id=DLGWJ6FAW36ZI4BFR5WFOV2RHXQDQSRFTQLVY64HCQYEZ6232O6A, seg_pos={0 0}, upload=262415]: Done uploading piece. err: <nil>
Aug 19 05:31:32.021360552 (252h5m53.980457787s) [noise=true , address=45.91.116.4:4941 , piece_id=XLHBRNY3ZMQRAJWGNRPDAA4AXTUEY4YQKLUMTLYWMDCZTGWMEQ6Q, seg_pos={0 0}, upload=262415]: Done uploading piece. err: <nil>
Aug 19 05:31:32.030444924 (252h5m53.989542163s) [noise=true , address=77.90.31.21:4890 , piece_id=4WLT7FEV5Q65YHAIBASU5G4POBW5QRFFQ4H3PHSBL6HR73ZH4XQA, seg_pos={0 0}, upload=262415]: Done uploading piece. err: <nil>
Aug 19 05:31:32.036025640 (252h5m53.995122873s) [noise=true , address=104.36.82.60:4342 , piece_id=CUUNPNCIKAVK6DYWN2D3WQNA2H7R2YOYBE7HMC56RMIBNRQY6TZA, seg_pos={0 0}, upload=262415]: Done uploading piece. err: <nil>
Aug 19 05:31:32.036150183 (252h5m53.995247415s) [noise=true , address=104.36.82.23:4446 , piece_id=GRZIXHV6UDYWCBSIPDTAAEGFN6VLR2T22B7XKFDQJE4WSHELLXPQ, seg_pos={0 0}, upload=262415]: Done uploading piece. err: <nil>
Aug 19 05:31:32.038054980 (252h5m53.997152232s) [noise=true , address=77.90.31.12:4621 , piece_id=NFUPTPMDUDBPT2QEBK42EB5NOVE73FYOYUIVNGIYS6ODRPUUFYWA, seg_pos={0 0}, upload=262415]: Done uploading piece. err: <nil>
Aug 19 05:31:32.038183592 (252h5m53.997280827s) [noise=true , address=77.90.31.13:4636 , piece_id=3DXYBNJMHLR53B24BCWC43WYGSV7ATFEQRYJM2WCEFORPRNUMY4Q, seg_pos={0 0}, upload=262415]: Done uploading piece. err: <nil>
Aug 19 05:31:32.051421311 (252h5m54.010518542s) [noise=true , address=79.127.219.47:4062 , piece_id=ALSAEQIIBVCA7T76QIBVQ7JXMDGP74RPNMAFLDSLDPFDBS6LNICQ, seg_pos={0 0}, upload=262415]: Done uploading piece. err: <nil>
Aug 19 05:31:32.054876511 (252h5m54.013973766s) [noise=true , address=104.36.82.34:4563 , piece_id=QZE2LRF4FOODIJXF7YAGKOBNMVPZYXKPVRADHXIFTGELF75BJIQA, seg_pos={0 0}, upload=262415]: Done uploading piece. err: <nil>
Aug 19 05:31:32.060103348 (252h5m54.019200587s) [noise=true , address=104.36.82.44:4895 , piece_id=TQHQYFVSIHONZNO6QBZORM3BBI4EJSQ75WF3BMTHFZAJJDKMRA7Q, seg_pos={0 0}, upload=262415]: Done uploading piece. err: <nil>
Aug 19 05:31:32.060125923 (252h5m54.019223162s) [seg_pos={0 0}, upload=262415]: Segment reached optimal threshold of 14 pieces.
Aug 19 05:31:32.060590479 (252h5m54.019687715s) [noise=true , address=79.127.219.38:4033 , piece_id=5MTCGC3URKN4MEQ7MKLMIRAOXPG3XIFOHFMTQENO46WTMJSCDVSA, seg_pos={0 0}, upload=262415]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uncFbYBwwVapsLrj3e4b12V6MzSjH9LWYxgWTB6q7JdYsDCpX): protocol: expected piece hash; context canceled; EOF
Aug 19 05:31:32.061022765 (252h5m54.020120018s) [noise=true , address=77.90.31.11:4573 , piece_id=RD5MDTHZMTDOAWN7E7K3FHLGG54XJI475WPC2LUZEH4MTUVP25SA, seg_pos={0 0}, upload=262415]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Q3atbNPC93bXM7pr4KCeea5La9sX4xs2qNwGxNWSrNQs1ftrV): protocol: expected piece hash; context canceled; EOF
Aug 19 05:31:32.061102216 (252h5m54.020199465s) [noise=true , address=79.127.205.245:4236 , piece_id=JMRUFAAZGXSLHLYVFUYLPPNJUOIK4PKQZR2DL3B5VCRP6GWBB5EA, seg_pos={0 0}, upload=262415]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fuP4WPd2tsi43vnWLWvNoEruDTrJpHt9TFMcsd6YJeFHknBd3): protocol: write tcp 139.59.144.22:43998->79.127.205.245:4236: use of closed network connection; write tcp 139.59.144.22:43998->79.127.205.245:4236: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43998->79.127.205.245:4236: use of closed network connection
Aug 19 05:31:32.061344003 (252h5m54.02044125s) [noise=true , address=45.91.116.12:4256 , piece_id=LSI63BLNFPZNM6AAMLQPAH4HEB25UV7UK4WY6BVTG3YP777ZRE5A, seg_pos={0 0}, upload=262415]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V53FpFLqgnXXEZSaUboULEwVpygzejqxEWdLgATj9MJVzacME): protocol: expected piece hash; context canceled; EOF
Aug 19 05:31:32.061379452 (252h5m54.020476702s) [seg_pos={0 0}, upload=262415]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 05:31:32.061497766 (252h5m54.020595013s) [upload=262415]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 05:31:32.061503481 (252h5m54.020600717s) [upload=262415]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 05:31:32.217477949 (252h5m54.176575194s) [upload=262415]: Done uploading segments...
Aug 19 05:31:32.217516166 (252h5m54.176613405s) [upload=262415]: Upload finished. err: <nil>
Duration: 1.652309222s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.....X.......X..............................................] duration:
811.985997ms
FAIL (2025-08-19 05:30:42 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 812.007701ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [......X.......X.............................................] duration:
847.164273ms
FAIL (2025-08-19 05:30:23 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 847.189906ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
2.370830902s
FAIL (2025-08-19 05:29:55 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.370872916s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.......X.......X............................................] duration:
826.689478ms
FAIL (2025-08-19 05:29:27 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 826.718167ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
2.424037509s
FAIL (2025-08-19 05:28:53 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.42407019s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [........X.......X...........................................] duration:
808.69872ms
FAIL (2025-08-19 05:28:28 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 808.798665ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.........X.......X..........................................] duration:
1.009950832s
FAIL (2025-08-19 05:28:11 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.00998248s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
2.480997962s
FAIL (2025-08-19 05:27:40 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.481026889s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..........X.......X.........................................] duration:
835.456955ms
FAIL (2025-08-19 05:27:29 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 835.490408ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
2.318560326s
FAIL (2025-08-19 05:26:53 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.318663397s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...........X.......X........................................] duration:
798.395272ms
FAIL (2025-08-19 05:26:27 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 798.430696ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
2.449714241s
FAIL (2025-08-19 05:25:59 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.449762688s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [............X.......X.......................................] duration:
833.155742ms
FAIL (2025-08-19 05:25:57 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 833.193736ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.............X.......X......................................] duration:
858.450737ms
FAIL (2025-08-19 05:25:23 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 858.576044ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..............X.......X.....................................] duration:
806.90233ms
FAIL (2025-08-19 05:25:05 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 806.953375ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
2.341713659s
FAIL (2025-08-19 05:24:59 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.341751985s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...............X.......X....................................] duration:
863.467512ms
FAIL (2025-08-19 05:24:03 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 863.522173ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
2.431827901s
FAIL (2025-08-19 05:24:00 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.431864876s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [................X.......X...................................] duration:
807.848201ms
FAIL (2025-08-19 05:23:17 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 807.898307ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
2.572818261s
FAIL (2025-08-19 05:22:55 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.572860532s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.................X.......X..................................] duration:
1.183422335s
FAIL (2025-08-19 05:22:25 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.183500838s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
2.496483412s
FAIL (2025-08-19 05:21:53 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.496516953s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..................X.......X.................................] duration:
982.114026ms
FAIL (2025-08-19 05:21:25 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 982.145585ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [X........X..................................................] duration:
2.307661356s
FAIL (2025-08-19 05:20:58 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.307714848s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...................X.......X................................] duration:
847.447317ms
FAIL (2025-08-19 05:20:17 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 847.478814ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.X........X.................................................] duration:
2.660141954s
FAIL (2025-08-19 05:20:12 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.660180292s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [....................X.......X...............................] duration:
900.958524ms
FAIL (2025-08-19 05:19:20 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 900.997416ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..X........X................................................] duration:
2.696232945s
FAIL (2025-08-19 05:19:18 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.696266774s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.....................X.......X..............................] duration:
820.452114ms
FAIL (2025-08-19 05:18:47 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 820.509794ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [......................X.......X.............................] duration:
943.223967ms
FAIL (2025-08-19 05:18:31 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 943.260832ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [X..X........X...............................................] duration:
2.37321634s
FAIL (2025-08-19 05:18:19 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.373272832s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.......................X.......X............................] duration:
830.271987ms
FAIL (2025-08-19 05:17:38 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 830.304816ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [X.......................X.......X...........................] duration:
863.106841ms
FAIL (2025-08-19 05:17:22 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 863.135137ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [.X..X........X..............................................] duration:
2.500469162s
FAIL (2025-08-19 05:17:09 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.500518645s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [..X..X........X.............................................] duration:
2.445093531s
FAIL (2025-08-19 05:16:40 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.445130724s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [XX.......................X.......X..........................] duration:
870.523655ms
FAIL (2025-08-19 05:16:28 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 870.578166ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [...X..X........X............................................] duration:
2.32796047s
FAIL (2025-08-19 05:16:13 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.327991057s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.XX.......................X.......X.........................] duration:
1.016248776s
FAIL (2025-08-19 05:16:11 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.0162801s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [..XX.......................X.......X........................] duration:
853.781639ms
FAIL (2025-08-19 05:15:41 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 853.811928ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [....X..X........X...........................................] duration:
2.241995896s
FAIL (2025-08-19 05:15:40 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.242031794s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [.....X..X........X..........................................] duration:
2.53710317s
FAIL (2025-08-19 05:15:13 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.537147071s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [X..XX.......................X.......X.......................] duration:
991.493887ms
FAIL (2025-08-19 05:14:45 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 991.521407ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [.X..XX.......................X.......X......................] duration:
838.354289ms
FAIL (2025-08-19 05:14:21 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 838.394763ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [......X..X........X.........................................] duration:
2.434412577s
FAIL (2025-08-19 05:14:13 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.434475616s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [..X..XX.......................X.......X.....................] duration:
839.248166ms
FAIL (2025-08-19 05:13:38 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 839.289105ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [.......X..X........X........................................] duration:
2.551877707s
FAIL (2025-08-19 05:13:29 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 04:41:30.078322559 (251h15m52.037419816s) [upload=261529]: Starting upload
Aug 19 04:41:30.078421183 (251h15m52.037518431s) [upload=261529]: Done starting upload
Aug 19 04:41:30.078641351 (251h15m52.037738598s) [upload=261529]: Uploading segments...
Aug 19 04:41:30.078816100 (251h15m52.037913352s) [upload=261529]: Got next segment. Inline: false
Aug 19 04:41:30.078872521 (251h15m52.037969778s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:41:30.078879668 (251h15m52.037976928s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:41:30.302310407 (251h15m52.261407662s) [seg_pos={0 0}, upload=261529]: Begin upload segment...
Aug 19 04:41:30.302400447 (251h15m52.261497679s) [seg_pos={0 0}, upload=261529]: Done begin upload segment.
Aug 19 04:41:30.302468752 (251h15m52.261566s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302484436 (251h15m52.261581689s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.302988125 (251h15m52.262085398s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303029071 (251h15m52.262126326s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303506390 (251h15m52.262603645s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303515915 (251h15m52.262613165s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303922040 (251h15m52.263019292s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.303939454 (251h15m52.263036708s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304344338 (251h15m52.26344159s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304359751 (251h15m52.263457003s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.304868259 (251h15m52.263965509s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305343476 (251h15m52.264440733s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.305838014 (251h15m52.26493527s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306178833 (251h15m52.265276088s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.306275080 (251h15m52.265372335s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307274175 (251h15m52.266371425s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.307830640 (251h15m52.266927895s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.308405302 (251h15m52.267502555s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Uploading piece...
Aug 19 04:41:30.869123149 (251h15m52.828220404s) [upload=261529]: Next returned nil segment
Aug 19 04:41:30.869149722 (251h15m52.828246979s) [upload=261529]: Waiting for error group managing segments...
Aug 19 04:41:31.015668725 (251h15m52.974765964s) [noise=true , address=77.90.31.21:4880 , piece_id=4RRNWORSBR6OZVAXDR7M5BZAUXHIZI5UG6M2T7KAZMPCFI3SPHQQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.028060630 (251h15m52.987157869s) [noise=true , address=104.36.82.8:4994 , piece_id=XDD4E3GOQ4CNVQV4MHLB4V5MKVCP6LXOSFPQFZ7DQIVAU2L3ADSQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.041949819 (251h15m53.001047065s) [noise=true , address=104.36.82.16:4321 , piece_id=QFCD24MKXEVZGYGYOS44PJNA7PG6FY73B36B3VVARP6EY7EIIWIA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.045425210 (251h15m53.004522455s) [noise=true , address=104.36.82.23:4446 , piece_id=E3PO3SWCHH76PLUFHQRRWOVZ6HDHVTQXMGA6KULBOA25B2CBUYJQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.046244392 (251h15m53.005341631s) [noise=true , address=104.36.82.6:4908 , piece_id=5INQZYMGVPL7V76SLDCYQGNUEWAMHM22DUEHOLNIEHRONB73FICQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.058007208 (251h15m53.017104454s) [noise=true , address=104.36.82.11:4125 , piece_id=2TN76LVES4DZJ2VBK7S7NUX5P25JRBI532YILGSKK75MPKWFUVXQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.066975728 (251h15m53.026072963s) [noise=true , address=104.36.82.18:4389 , piece_id=TURWX5FFYC7YAS322VUG5VHZZKM3H275NG5UOHRCCUH6CDZGQS5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070484936 (251h15m53.029582171s) [noise=true , address=77.90.31.15:4726 , piece_id=AAJUJ7IGOABNZBCJQQMERI7QFZCLQC3NH7XHTBRLRNZSOAFAEARQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.070956309 (251h15m53.030053541s) [noise=true , address=45.91.116.11:4138 , piece_id=C5YAWMDBSCOCK6YZB4EL6QBX5R5W3EQV322FXNNYCMFP3B7ZB7MQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.075660693 (251h15m53.034757927s) [noise=true , address=104.36.82.41:4769 , piece_id=2LZEO3CCCDAPW2KPGTAUF4MVHBF35DQ53OHRLAVQMXBJSL6H7RBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.109127214 (251h15m53.068224466s) [noise=true , address=77.90.31.12:4616 , piece_id=AQL6KHFJDAZUCPGUKP7ZQXWIS4MO2SOXMDX2URELIIDDE3VWWZBQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.124116672 (251h15m53.083213919s) [noise=true , address=104.36.82.60:4364 , piece_id=XS56FKJDEAFBDJ3QZP2I4ZLJ7XSVHNYGG7NX7BASE7CQ7ZBUETFA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.185697296 (251h15m53.144794542s) [noise=true , address=77.90.31.11:4565 , piece_id=B57447OCSCXZ3BQAIOVRU7VC4SSJAWLORX7LEOSBQISDGY2C4MOQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204696000 (251h15m53.16379329s) [noise=true , address=77.90.31.17:4776 , piece_id=7EV5PHAX7OBVAZ2G6NL3ZP6MFUAF4JPC4KUSSRTPKFS5FD3SNBNA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: <nil>
Aug 19 04:41:31.204737159 (251h15m53.163834407s) [seg_pos={0 0}, upload=261529]: Segment reached optimal threshold of 14 pieces.
Aug 19 04:41:31.205040408 (251h15m53.164137659s) [noise=true , address=77.90.31.16:4754 , piece_id=5AM3X6JHPJPUV24B7RLFEFCOOVDCR4HJPEXYAEV2T46AAAHVD4DA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205172276 (251h15m53.164269527s) [noise=true , address=79.127.201.215:4356 , piece_id=4XQGB22327W3RHX7S5HG72WEPNSG3IJHSL5G6PJ4AWRXN7HFZZMQ, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vCjcquqBhLnXgnGVd5D3j7x8LBa7RrHngfwm2EA3eRJ2Mb35o): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205236612 (251h15m53.164333857s) [noise=true , address=66.23.193.175:4554 , piece_id=U7CNDJJ5YPOZMXZOD7WUCFAAOFDGZVTGKJ6T7AB3H42QLJI2FL5Q, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48524->66.23.193.175:4554: use of closed network connection
Aug 19 04:41:31.205287570 (251h15m53.164384819s) [noise=true , address=77.90.31.19:4876 , piece_id=BB6FIXMYP2VAJO4ZD3OUSC22F34XFZUVMQRCAHUXVPWMGRRMQLSA, seg_pos={0 0}, upload=261529]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 04:41:31.205316044 (251h15m53.164413302s) [seg_pos={0 0}, upload=261529]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 04:41:31.205410529 (251h15m53.164507785s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 04:41:31.205416764 (251h15m53.164514019s) [upload=261529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 04:41:31.422937502 (251h15m53.382034737s) [upload=261529]: Done uploading segments...
Aug 19 04:41:31.422958907 (251h15m53.382056145s) [upload=261529]: Upload finished. err: <nil>
Duration: 2.551987383s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [...X..XX.......................X.......X....................] duration:
943.694642ms
FAIL (2025-08-19 05:13:15 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 943.728084ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [....X..XX.......................X.......X...................] duration:
818.583047ms
FAIL (2025-08-19 05:12:44 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:59:45.218769376 (251h34m7.177866622s) [upload=261847]: Starting upload Aug 19 04:59:45.218840532 (251h34m7.177937777s) [upload=261847]: Done starting upload Aug 19 04:59:45.218938430 (251h34m7.178035692s) [upload=261847]: Uploading segments... Aug 19 04:59:45.219026153 (251h34m7.178123434s) [upload=261847]: Got next segment. Inline: true Aug 19 04:59:45.219034256 (251h34m7.178131499s) [upload=261847]: Waiting for error group managing segments... Aug 19 04:59:45.219100783 (251h34m7.178198037s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 04:59:45.219107454 (251h34m7.178204701s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 19 04:59:45.219112163 (251h34m7.178209411s) [upload=261847]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 19 04:59:45.363002600 (251h34m7.322099846s) [upload=261847]: Done uploading segments... Aug 19 04:59:45.363042825 (251h34m7.322140073s) [upload=261847]: Upload finished. err: metaclient: Too Many Requests
Duration: 818.634191ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|