Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........X..........................X.....................] duration:
2.423083994s
FAIL (2025-08-09 09:05:29 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.423123984s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [............X..........................X....................] duration:
2.375053287s
FAIL (2025-08-09 09:05:01 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.375108878s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
842.767973ms
FAIL (2025-08-09 09:04:47 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 844.48648ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............X..........................X...................] duration:
2.420552529s
FAIL (2025-08-09 09:04:13 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.420587042s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
834.7273ms
FAIL (2025-08-09 09:04:08 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 834.811282ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
824.715813ms
FAIL (2025-08-09 09:03:13 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 824.74942ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............X..........................X..................] duration:
2.507545395s
FAIL (2025-08-09 09:03:02 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.507621614s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
841.848781ms
FAIL (2025-08-09 09:02:44 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 841.878168ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............X..........................X.................] duration:
2.622397914s
FAIL (2025-08-09 09:02:17 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.622431743s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
870.578711ms
FAIL (2025-08-09 09:01:52 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 870.628311ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [................X..........................X................] duration:
2.490177681s
FAIL (2025-08-09 09:01:31 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.490213628s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
874.340615ms
FAIL (2025-08-09 09:00:48 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 874.370733ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................X..........................X...............] duration:
2.24875283s
FAIL (2025-08-09 09:00:44 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.248790143s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
856.590741ms
FAIL (2025-08-09 09:00:28 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 856.626133ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..................X..........................X..............] duration:
2.332504324s
FAIL (2025-08-09 08:59:38 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.332536887s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [X.....................................X.....................] duration:
930.2277ms
FAIL (2025-08-09 08:59:27 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 930.27696ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.X.....................................X....................] duration:
845.41238ms
FAIL (2025-08-09 08:58:53 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 845.43652ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................X..........................X.............] duration:
2.444073591s
FAIL (2025-08-09 08:58:31 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.44410236s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..X.....................................X...................] duration:
878.257889ms
FAIL (2025-08-09 08:58:20 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 878.293028ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...X.....................................X..................] duration:
856.704762ms
FAIL (2025-08-09 08:58:02 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 856.739455ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................X..........................X............] duration:
2.3976618s
FAIL (2025-08-09 08:57:48 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.397704133s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....................X..........................X...........] duration:
2.40020777s
FAIL (2025-08-09 08:57:23 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.400238754s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [....X.....................................X.................] duration:
848.277301ms
FAIL (2025-08-09 08:57:08 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 848.306233ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.....X.....................................X................] duration:
810.004989ms
FAIL (2025-08-09 08:56:51 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 810.051556ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [......................X..........................X..........] duration:
2.399711163s
FAIL (2025-08-09 08:56:44 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.399796022s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......................X..........................X.........] duration:
2.400253923s
FAIL (2025-08-09 08:56:03 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.400288757s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [......X.....................................X...............] duration:
880.128198ms
FAIL (2025-08-09 08:55:59 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 880.168913ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [........................X..........................X........] duration:
2.358522925s
FAIL (2025-08-09 08:55:19 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.358552904s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.......X.....................................X..............] duration:
915.68197ms
FAIL (2025-08-09 08:54:48 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 915.702427ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........................X..........................X.......] duration:
2.447950518s
FAIL (2025-08-09 08:54:43 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.447977516s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................X..........................X......] duration:
2.404346149s
FAIL (2025-08-09 08:53:41 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.404377586s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [........X.....................................X.............] duration:
820.721224ms
FAIL (2025-08-09 08:53:39 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 820.747882ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........................X..........................X.....] duration:
2.570184722s
FAIL (2025-08-09 08:52:57 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.570214839s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.........X.....................................X............] duration:
845.393209ms
FAIL (2025-08-09 08:52:37 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 845.436147ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [............................X..........................X....] duration:
2.297322135s
FAIL (2025-08-09 08:52:05 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.297355659s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..........X.....................................X...........] duration:
856.127614ms
FAIL (2025-08-09 08:52:03 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 856.163347ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...........X.....................................X..........] duration:
845.697214ms
FAIL (2025-08-09 08:51:21 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 845.730784ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............................X..........................X...] duration:
2.359323568s
FAIL (2025-08-09 08:51:11 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.35935271s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [............X.....................................X.........] duration:
820.063261ms
FAIL (2025-08-09 08:50:53 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 820.100435ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............................X..........................X..] duration:
2.393072782s
FAIL (2025-08-09 08:50:07 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.39311025s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............................X..........................X.] duration:
2.450327413s
FAIL (2025-08-09 08:49:51 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 2.450359363s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.............X.....................................X........] duration:
1.952154301s
FAIL (2025-08-09 08:49:50 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.952200973s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [................................X..........................X] duration:
1.607983109s
FAIL (2025-08-09 08:49:28 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 08:49:26.743685689 (15h23m48.702782924s) [upload=16007]: Starting upload
Aug 9 08:49:26.743732418 (15h23m48.702829672s) [upload=16007]: Done starting upload
Aug 9 08:49:26.743990417 (15h23m48.703087654s) [upload=16007]: Uploading segments...
Aug 9 08:49:26.744040895 (15h23m48.70313815s) [upload=16007]: Got next segment. Inline: false
Aug 9 08:49:26.744077868 (15h23m48.703175115s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:49:26.744083460 (15h23m48.703180706s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:49:26.944316786 (15h23m48.903414031s) [seg_pos={0 0}, upload=16007]: Begin upload segment...
Aug 9 08:49:26.944371440 (15h23m48.903468689s) [seg_pos={0 0}, upload=16007]: Done begin upload segment.
Aug 9 08:49:26.944401234 (15h23m48.903498482s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944415760 (15h23m48.903512996s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944793479 (15h23m48.903890726s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.944936732 (15h23m48.904033984s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945136285 (15h23m48.904233538s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.945956063 (15h23m48.905053441s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946177462 (15h23m48.90527471s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946910704 (15h23m48.906007955s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.946998295 (15h23m48.906095546s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947877980 (15h23m48.906975234s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.947979184 (15h23m48.907076435s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948953778 (15h23m48.908051035s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.948999082 (15h23m48.908096331s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949549789 (15h23m48.908647035s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.949998421 (15h23m48.909095668s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.950998232 (15h23m48.910095493s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951472514 (15h23m48.910569762s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:26.951944136 (15h23m48.911041391s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Uploading piece...
Aug 9 08:49:27.520100575 (15h23m49.479197839s) [upload=16007]: Next returned nil segment
Aug 9 08:49:27.520130851 (15h23m49.479228096s) [upload=16007]: Waiting for error group managing segments...
Aug 9 08:49:27.657666493 (15h23m49.616763724s) [noise=true , address=45.91.116.10:4084 , piece_id=OD7264UM3YUMMM2Y5TYYBM4D7MKJAMQLVASK2K77QQSRNQ4BWZ2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.669387627 (15h23m49.628484874s) [noise=true , address=104.36.82.44:4897 , piece_id=VI4POTILK3XDJPZKWOY4F3CM2I7FG3CYGL56FXPIKE7G5RDZQ4RA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.673970532 (15h23m49.633067779s) [noise=true , address=104.36.82.34:4573 , piece_id=KY4UJDCRZURF3NZGV7HJXDA5PPAV7L2O4UCYBSLMMCX34YMJFLIA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.676405178 (15h23m49.635502417s) [noise=true , address=104.36.82.13:4188 , piece_id=M5NRNLYBXA5SX6RGSTR3Y4NZTPGZWWVUVL6LTIVPHOHIQUFQGNCA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.679900463 (15h23m49.638997702s) [noise=true , address=104.36.82.62:4431 , piece_id=7UQDU6SNVMRBQ4NFQPR4AYKWNZL7RZC673ZNKWY325CMNIP7RY3Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.681806158 (15h23m49.640903396s) [noise=true , address=104.36.82.16:4313 , piece_id=YERZMVUBDKCSH5Q63UYGFLPCWG734PRN6CZBNJ5BN6NVDNTI72OA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.687709835 (15h23m49.646807084s) [noise=true , address=104.36.82.37:4659 , piece_id=2NCXAJSGMLUAE4D46JWI6FTUJLORW4VBMU6Q35OJ4DMCLSUR3ZWA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.689052504 (15h23m49.648149751s) [noise=true , address=104.36.82.45:4917 , piece_id=VTNQLXRITCAU7YG6DT7HITHTG4Y3BWNMX4VCKVYWFQIJGHXZL5IQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694600137 (15h23m49.653697372s) [noise=true , address=104.36.82.46:4512 , piece_id=JYOAFSKPEG64WEN4BNRNX2SEJOMLAQ66YY5YEP5CNTUMHATDNNMQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.694964630 (15h23m49.654061869s) [noise=true , address=104.36.82.19:4431 , piece_id=BFF2MVTAYYEICHIJHPZRYRQYQRAFYDVWQEDVS4K6FEJWUAZPLIAQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699453495 (15h23m49.658550733s) [noise=true , address=104.36.82.6:4881 , piece_id=6OC4UGWJG52AHYOKDZIEOTTER6T3S3ZBAQ7DGVY6QHIXRSEVIQDQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.699550339 (15h23m49.658647575s) [noise=true , address=104.36.82.22:4523 , piece_id=HIGJ42WHSKLWFJJTBYFZDJ53J5KCKBODI4PEFA362MWRPBV7HPJQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.701570687 (15h23m49.660667924s) [noise=true , address=104.36.82.16:4325 , piece_id=FKDO5VK32MCIPCBHIPAAG3IXWVO5OU4PK4SJI7HKNMRRW6R5SOGQ, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711406591 (15h23m49.670503837s) [noise=true , address=104.36.82.36:4625 , piece_id=JYMBQDLHDQKXXLSEWGUMY754IIUPH5COBQ3C3EYO4IBJPJXJGT4Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: <nil>
Aug 9 08:49:27.711443385 (15h23m49.670540635s) [seg_pos={0 0}, upload=16007]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:49:27.711656354 (15h23m49.670753601s) [noise=true , address=104.36.82.58:4275 , piece_id=H7ULUBWFFAPH7IXW6UGBWYDHVNXJTBREXJOXKXP7OKAE5M6V243Q, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711722267 (15h23m49.670819499s) [noise=true , address=64.49.13.137:4497 , piece_id=YLFE57Q6X5PZKEGRDJW5IHTX45GV6GLAGKBFTHBFGWT6ICXC4J6A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qm8sngG152bw9bzNn2cTN3psYcMBDVbBTewLFPWVhiNhcczkm): protocol: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59072->64.49.13.137:4497: use of closed network connection
Aug 9 08:49:27.711799933 (15h23m49.670897177s) [noise=true , address=104.36.82.60:4352 , piece_id=D6HFGVCUDJEKF6DUPXV5WHWON5MXXBWCOHBPYQXF7NA6CIVTGVSA, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711817735 (15h23m49.670914981s) [noise=true , address=104.36.82.38:4722 , piece_id=DGWYFCYYJFMAULKVVMK6B3A4PFB6Z63VRSQBMOT3ZVPMXKXESE2A, seg_pos={0 0}, upload=16007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc): protocol: expected piece hash; context canceled; EOF
Aug 9 08:49:27.711830985 (15h23m49.670928234s) [seg_pos={0 0}, upload=16007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:49:27.711925842 (15h23m49.671023099s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:49:27.711932536 (15h23m49.671029795s) [upload=16007]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:49:27.870375403 (15h23m49.829472677s) [upload=16007]: Done uploading segments...
Aug 9 08:49:27.870462629 (15h23m49.829559909s) [upload=16007]: Upload finished. err: <nil>
Duration: 1.608013782s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..............X.....................................X.......] duration:
887.055188ms
FAIL (2025-08-09 08:48:50 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 887.080811ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
2.364474472s
FAIL (2025-08-09 08:48:20 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
Aug 9 08:28:13.240669419 (15h2m35.199766665s) [upload=15638]: Starting upload
Aug 9 08:28:13.240733944 (15h2m35.199831192s) [upload=15638]: Done starting upload
Aug 9 08:28:13.240872982 (15h2m35.199970234s) [upload=15638]: Uploading segments...
Aug 9 08:28:13.241079499 (15h2m35.200176745s) [upload=15638]: Got next segment. Inline: false
Aug 9 08:28:13.241124567 (15h2m35.200221826s) [upload=15638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:28:13.241161881 (15h2m35.200259139s) [upload=15638]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:28:13.438488388 (15h2m35.397585691s) [seg_pos={0 0}, upload=15638]: Begin upload segment...
Aug 9 08:28:13.438557720 (15h2m35.397654968s) [seg_pos={0 0}, upload=15638]: Done begin upload segment.
Aug 9 08:28:13.438585890 (15h2m35.397683167s) [noise=true , address=104.36.82.60:4345 , piece_id=WTUSWXUTY4KLXVULDLGKIYL4L6VYL2W6P3HB7W3L3KJIKY6N2NJA, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.438633294 (15h2m35.397730541s) [noise=true , address=77.90.31.19:4872 , piece_id=DU3DZDYSFZG2UQ57QZTYMUZDUCAA443KON37AHMQWET2NJYQWZ6A, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.439039750 (15h2m35.398137001s) [noise=true , address=64.49.13.169:4706 , piece_id=S2YHCTQ5JCTOYDF7C57HZV74K3GFVBFDKP3GUXNBGZOFY3V7XB2Q, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.439054845 (15h2m35.39815209s) [noise=true , address=104.36.82.13:4218 , piece_id=LY2S2YJRPIOVU2NB3JXF7UUX6T4KSVYAJMYVJN5YPAG4FC5BBCFQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.439326357 (15h2m35.398423595s) [noise=true , address=104.36.82.58:4289 , piece_id=EHP7SPBB2346TPWTDJQXGHGGUKXQFKC4QANJFQQUQQOR47NO3IPQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.439434877 (15h2m35.398532129s) [noise=true , address=104.36.82.45:4925 , piece_id=CKPIOOGSRLW6QBQDM2OLQUPNE2BT5YC4EMMDCL3WLTLIMY7SGGZQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.440101858 (15h2m35.399199139s) [noise=true , address=104.36.82.35:4610 , piece_id=KACQNESKG677EBPXOHRRXDAP56EHWV35HKTZQZEY72F4N2T7TS2Q, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.440525550 (15h2m35.399622784s) [noise=true , address=104.36.82.3:4795 , piece_id=Z6E2LED6BNUPDHH4DWHI3MQTYI64XHTMY7ANDSVXSFMGKUS2YPRQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.441368342 (15h2m35.40046559s) [noise=true , address=104.36.82.61:4375 , piece_id=L5YP7MNLUJK7LVW27HUXFKQW4CDNYV6OT44YRJC2LBSCZOQFIT4A, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.441869558 (15h2m35.400966813s) [noise=true , address=104.36.82.18:4385 , piece_id=HXYMSF7VFKZLQ7EDVCBMRNIWWEHJS3JGVRVDJXQNIPVFIPZWH7VQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.442227603 (15h2m35.40132485s) [noise=true , address=104.36.82.23:4470 , piece_id=XTXQA64XHMNOQGOPSYBOHHYKPNXAM2QSNC6ZXFIBOOZOWYI7HFWQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.442628619 (15h2m35.401725871s) [noise=true , address=104.36.82.46:4503 , piece_id=4ZBFEDGEHFA7K76WDNDYT5YMHPLYGP7W4YIYAPV6T7GWPZDKBD7Q, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.442914820 (15h2m35.402012073s) [noise=true , address=64.49.13.26:4946 , piece_id=J4G6XFNX6ZIGBMHZF2Z3N37SD3HTTGB6FWZOJOVGJIC2PDDTJSKQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.443213885 (15h2m35.402311131s) [noise=true , address=104.36.82.54:4176 , piece_id=2G2LW6WR2I5B5ERD3TJQL5WS5TBTPHS6S3O2JZY4USOBXEC53C6A, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.443428267 (15h2m35.402525513s) [noise=true , address=104.36.82.62:4412 , piece_id=ADYVUQ57QEEDZ2FDGPR3C2EKHI4RWOE4U6VRKDHCF3IC3QR2LK2A, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.443813452 (15h2m35.402910703s) [noise=true , address=104.36.82.15:4291 , piece_id=LRAYTCPUN6APCBAE4U4TRHHSPTNDCDUPWK2CF64FOKTKLUDYNWNQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.444139434 (15h2m35.403236681s) [noise=true , address=104.36.82.6:4891 , piece_id=7HPMVRYVJXMRDAZSY73CT2GK3YM54BYLVCQ6NRIH7C7U7K7OXEQA, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.444576027 (15h2m35.403673276s) [noise=true , address=104.36.82.17:4345 , piece_id=UNNJ3JD4VDZ74N52C5G6QTVJFG2JONR25XTR32F6ZZFZDUH34QEA, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:14.026403017 (15h2m35.985500265s) [upload=15638]: Next returned nil segment
Aug 9 08:28:14.026499168 (15h2m35.985596416s) [upload=15638]: Waiting for error group managing segments...
Aug 9 08:28:14.178012028 (15h2m36.137109273s) [noise=true , address=77.90.31.19:4872 , piece_id=DU3DZDYSFZG2UQ57QZTYMUZDUCAA443KON37AHMQWET2NJYQWZ6A, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.182219420 (15h2m36.141316665s) [noise=true , address=104.36.82.62:4412 , piece_id=ADYVUQ57QEEDZ2FDGPR3C2EKHI4RWOE4U6VRKDHCF3IC3QR2LK2A, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.182901333 (15h2m36.141998578s) [noise=true , address=104.36.82.13:4218 , piece_id=LY2S2YJRPIOVU2NB3JXF7UUX6T4KSVYAJMYVJN5YPAG4FC5BBCFQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.183653520 (15h2m36.142750765s) [noise=true , address=104.36.82.6:4891 , piece_id=7HPMVRYVJXMRDAZSY73CT2GK3YM54BYLVCQ6NRIH7C7U7K7OXEQA, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.183712240 (15h2m36.142809486s) [noise=true , address=104.36.82.46:4503 , piece_id=4ZBFEDGEHFA7K76WDNDYT5YMHPLYGP7W4YIYAPV6T7GWPZDKBD7Q, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.184021408 (15h2m36.143118654s) [noise=true , address=104.36.82.23:4470 , piece_id=XTXQA64XHMNOQGOPSYBOHHYKPNXAM2QSNC6ZXFIBOOZOWYI7HFWQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.184177333 (15h2m36.143274578s) [noise=true , address=104.36.82.45:4925 , piece_id=CKPIOOGSRLW6QBQDM2OLQUPNE2BT5YC4EMMDCL3WLTLIMY7SGGZQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.184556211 (15h2m36.143653457s) [noise=true , address=104.36.82.3:4795 , piece_id=Z6E2LED6BNUPDHH4DWHI3MQTYI64XHTMY7ANDSVXSFMGKUS2YPRQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.184608088 (15h2m36.143705333s) [noise=true , address=104.36.82.35:4610 , piece_id=KACQNESKG677EBPXOHRRXDAP56EHWV35HKTZQZEY72F4N2T7TS2Q, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.186672666 (15h2m36.145769911s) [noise=true , address=104.36.82.18:4385 , piece_id=HXYMSF7VFKZLQ7EDVCBMRNIWWEHJS3JGVRVDJXQNIPVFIPZWH7VQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.188271372 (15h2m36.147368618s) [noise=true , address=104.36.82.60:4345 , piece_id=WTUSWXUTY4KLXVULDLGKIYL4L6VYL2W6P3HB7W3L3KJIKY6N2NJA, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.190035139 (15h2m36.149132384s) [noise=true , address=104.36.82.61:4375 , piece_id=L5YP7MNLUJK7LVW27HUXFKQW4CDNYV6OT44YRJC2LBSCZOQFIT4A, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.190829073 (15h2m36.149926319s) [noise=true , address=104.36.82.17:4345 , piece_id=UNNJ3JD4VDZ74N52C5G6QTVJFG2JONR25XTR32F6ZZFZDUH34QEA, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.191271905 (15h2m36.150369141s) [noise=true , address=104.36.82.15:4291 , piece_id=LRAYTCPUN6APCBAE4U4TRHHSPTNDCDUPWK2CF64FOKTKLUDYNWNQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.191281775 (15h2m36.150379011s) [seg_pos={0 0}, upload=15638]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:28:14.191581487 (15h2m36.150678723s) [noise=true , address=104.36.82.54:4176 , piece_id=2G2LW6WR2I5B5ERD3TJQL5WS5TBTPHS6S3O2JZY4USOBXEC53C6A, seg_pos={0 0}, upload=15638]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Yi5AibNiCNV8vUb9MKgooHCw8Nr79m9HqTrkrYybNZmatg9PD): protocol: expected piece hash; context canceled; EOF
Aug 9 08:28:14.191655070 (15h2m36.150752305s) [noise=true , address=104.36.82.58:4289 , piece_id=EHP7SPBB2346TPWTDJQXGHGGUKXQFKC4QANJFQQUQQOR47NO3IPQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v4QJpSy7hk4pShMu3FswnZAf3rzyCNUx5sUZSPSvxpecY8T9X): protocol: expected piece hash; context canceled; EOF
Aug 9 08:28:14.191766681 (15h2m36.150863917s) [noise=true , address=64.49.13.169:4706 , piece_id=S2YHCTQ5JCTOYDF7C57HZV74K3GFVBFDKP3GUXNBGZOFY3V7XB2Q, seg_pos={0 0}, upload=15638]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 139.59.144.22:57046->64.49.13.169:4706: use of closed network connection; write tcp 139.59.144.22:57046->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57046->64.49.13.169:4706: use of closed network connection
Aug 9 08:28:14.191834982 (15h2m36.150932218s) [noise=true , address=64.49.13.26:4946 , piece_id=J4G6XFNX6ZIGBMHZF2Z3N37SD3HTTGB6FWZOJOVGJIC2PDDTJSKQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hdE48UnygY1dCRASR39FkumY2V8f5Xb68Pd1WJ4AMLUnnGAfT): protocol: write tcp 139.59.144.22:39096->64.49.13.26:4946: use of closed network connection; write tcp 139.59.144.22:39096->64.49.13.26:4946: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39096->64.49.13.26:4946: use of closed network connection
Aug 9 08:28:14.191848663 (15h2m36.1509459s) [seg_pos={0 0}, upload=15638]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:28:14.191916508 (15h2m36.151013757s) [upload=15638]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:28:14.191921239 (15h2m36.151018488s) [upload=15638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:28:15.172074926 (15h2m37.131172172s) [upload=15638]: Done uploading segments...
Aug 9 08:28:15.172095781 (15h2m37.131193026s) [upload=15638]: Upload finished. err: <nil>
Duration: 2.364517321s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
2.335352657s
FAIL (2025-08-09 08:47:45 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
Aug 9 08:28:13.240669419 (15h2m35.199766665s) [upload=15638]: Starting upload
Aug 9 08:28:13.240733944 (15h2m35.199831192s) [upload=15638]: Done starting upload
Aug 9 08:28:13.240872982 (15h2m35.199970234s) [upload=15638]: Uploading segments...
Aug 9 08:28:13.241079499 (15h2m35.200176745s) [upload=15638]: Got next segment. Inline: false
Aug 9 08:28:13.241124567 (15h2m35.200221826s) [upload=15638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:28:13.241161881 (15h2m35.200259139s) [upload=15638]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:28:13.438488388 (15h2m35.397585691s) [seg_pos={0 0}, upload=15638]: Begin upload segment...
Aug 9 08:28:13.438557720 (15h2m35.397654968s) [seg_pos={0 0}, upload=15638]: Done begin upload segment.
Aug 9 08:28:13.438585890 (15h2m35.397683167s) [noise=true , address=104.36.82.60:4345 , piece_id=WTUSWXUTY4KLXVULDLGKIYL4L6VYL2W6P3HB7W3L3KJIKY6N2NJA, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.438633294 (15h2m35.397730541s) [noise=true , address=77.90.31.19:4872 , piece_id=DU3DZDYSFZG2UQ57QZTYMUZDUCAA443KON37AHMQWET2NJYQWZ6A, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.439039750 (15h2m35.398137001s) [noise=true , address=64.49.13.169:4706 , piece_id=S2YHCTQ5JCTOYDF7C57HZV74K3GFVBFDKP3GUXNBGZOFY3V7XB2Q, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.439054845 (15h2m35.39815209s) [noise=true , address=104.36.82.13:4218 , piece_id=LY2S2YJRPIOVU2NB3JXF7UUX6T4KSVYAJMYVJN5YPAG4FC5BBCFQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.439326357 (15h2m35.398423595s) [noise=true , address=104.36.82.58:4289 , piece_id=EHP7SPBB2346TPWTDJQXGHGGUKXQFKC4QANJFQQUQQOR47NO3IPQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.439434877 (15h2m35.398532129s) [noise=true , address=104.36.82.45:4925 , piece_id=CKPIOOGSRLW6QBQDM2OLQUPNE2BT5YC4EMMDCL3WLTLIMY7SGGZQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.440101858 (15h2m35.399199139s) [noise=true , address=104.36.82.35:4610 , piece_id=KACQNESKG677EBPXOHRRXDAP56EHWV35HKTZQZEY72F4N2T7TS2Q, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.440525550 (15h2m35.399622784s) [noise=true , address=104.36.82.3:4795 , piece_id=Z6E2LED6BNUPDHH4DWHI3MQTYI64XHTMY7ANDSVXSFMGKUS2YPRQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.441368342 (15h2m35.40046559s) [noise=true , address=104.36.82.61:4375 , piece_id=L5YP7MNLUJK7LVW27HUXFKQW4CDNYV6OT44YRJC2LBSCZOQFIT4A, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.441869558 (15h2m35.400966813s) [noise=true , address=104.36.82.18:4385 , piece_id=HXYMSF7VFKZLQ7EDVCBMRNIWWEHJS3JGVRVDJXQNIPVFIPZWH7VQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.442227603 (15h2m35.40132485s) [noise=true , address=104.36.82.23:4470 , piece_id=XTXQA64XHMNOQGOPSYBOHHYKPNXAM2QSNC6ZXFIBOOZOWYI7HFWQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.442628619 (15h2m35.401725871s) [noise=true , address=104.36.82.46:4503 , piece_id=4ZBFEDGEHFA7K76WDNDYT5YMHPLYGP7W4YIYAPV6T7GWPZDKBD7Q, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.442914820 (15h2m35.402012073s) [noise=true , address=64.49.13.26:4946 , piece_id=J4G6XFNX6ZIGBMHZF2Z3N37SD3HTTGB6FWZOJOVGJIC2PDDTJSKQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.443213885 (15h2m35.402311131s) [noise=true , address=104.36.82.54:4176 , piece_id=2G2LW6WR2I5B5ERD3TJQL5WS5TBTPHS6S3O2JZY4USOBXEC53C6A, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.443428267 (15h2m35.402525513s) [noise=true , address=104.36.82.62:4412 , piece_id=ADYVUQ57QEEDZ2FDGPR3C2EKHI4RWOE4U6VRKDHCF3IC3QR2LK2A, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.443813452 (15h2m35.402910703s) [noise=true , address=104.36.82.15:4291 , piece_id=LRAYTCPUN6APCBAE4U4TRHHSPTNDCDUPWK2CF64FOKTKLUDYNWNQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.444139434 (15h2m35.403236681s) [noise=true , address=104.36.82.6:4891 , piece_id=7HPMVRYVJXMRDAZSY73CT2GK3YM54BYLVCQ6NRIH7C7U7K7OXEQA, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.444576027 (15h2m35.403673276s) [noise=true , address=104.36.82.17:4345 , piece_id=UNNJ3JD4VDZ74N52C5G6QTVJFG2JONR25XTR32F6ZZFZDUH34QEA, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:14.026403017 (15h2m35.985500265s) [upload=15638]: Next returned nil segment
Aug 9 08:28:14.026499168 (15h2m35.985596416s) [upload=15638]: Waiting for error group managing segments...
Aug 9 08:28:14.178012028 (15h2m36.137109273s) [noise=true , address=77.90.31.19:4872 , piece_id=DU3DZDYSFZG2UQ57QZTYMUZDUCAA443KON37AHMQWET2NJYQWZ6A, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.182219420 (15h2m36.141316665s) [noise=true , address=104.36.82.62:4412 , piece_id=ADYVUQ57QEEDZ2FDGPR3C2EKHI4RWOE4U6VRKDHCF3IC3QR2LK2A, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.182901333 (15h2m36.141998578s) [noise=true , address=104.36.82.13:4218 , piece_id=LY2S2YJRPIOVU2NB3JXF7UUX6T4KSVYAJMYVJN5YPAG4FC5BBCFQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.183653520 (15h2m36.142750765s) [noise=true , address=104.36.82.6:4891 , piece_id=7HPMVRYVJXMRDAZSY73CT2GK3YM54BYLVCQ6NRIH7C7U7K7OXEQA, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.183712240 (15h2m36.142809486s) [noise=true , address=104.36.82.46:4503 , piece_id=4ZBFEDGEHFA7K76WDNDYT5YMHPLYGP7W4YIYAPV6T7GWPZDKBD7Q, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.184021408 (15h2m36.143118654s) [noise=true , address=104.36.82.23:4470 , piece_id=XTXQA64XHMNOQGOPSYBOHHYKPNXAM2QSNC6ZXFIBOOZOWYI7HFWQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.184177333 (15h2m36.143274578s) [noise=true , address=104.36.82.45:4925 , piece_id=CKPIOOGSRLW6QBQDM2OLQUPNE2BT5YC4EMMDCL3WLTLIMY7SGGZQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.184556211 (15h2m36.143653457s) [noise=true , address=104.36.82.3:4795 , piece_id=Z6E2LED6BNUPDHH4DWHI3MQTYI64XHTMY7ANDSVXSFMGKUS2YPRQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.184608088 (15h2m36.143705333s) [noise=true , address=104.36.82.35:4610 , piece_id=KACQNESKG677EBPXOHRRXDAP56EHWV35HKTZQZEY72F4N2T7TS2Q, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.186672666 (15h2m36.145769911s) [noise=true , address=104.36.82.18:4385 , piece_id=HXYMSF7VFKZLQ7EDVCBMRNIWWEHJS3JGVRVDJXQNIPVFIPZWH7VQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.188271372 (15h2m36.147368618s) [noise=true , address=104.36.82.60:4345 , piece_id=WTUSWXUTY4KLXVULDLGKIYL4L6VYL2W6P3HB7W3L3KJIKY6N2NJA, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.190035139 (15h2m36.149132384s) [noise=true , address=104.36.82.61:4375 , piece_id=L5YP7MNLUJK7LVW27HUXFKQW4CDNYV6OT44YRJC2LBSCZOQFIT4A, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.190829073 (15h2m36.149926319s) [noise=true , address=104.36.82.17:4345 , piece_id=UNNJ3JD4VDZ74N52C5G6QTVJFG2JONR25XTR32F6ZZFZDUH34QEA, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.191271905 (15h2m36.150369141s) [noise=true , address=104.36.82.15:4291 , piece_id=LRAYTCPUN6APCBAE4U4TRHHSPTNDCDUPWK2CF64FOKTKLUDYNWNQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.191281775 (15h2m36.150379011s) [seg_pos={0 0}, upload=15638]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:28:14.191581487 (15h2m36.150678723s) [noise=true , address=104.36.82.54:4176 , piece_id=2G2LW6WR2I5B5ERD3TJQL5WS5TBTPHS6S3O2JZY4USOBXEC53C6A, seg_pos={0 0}, upload=15638]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Yi5AibNiCNV8vUb9MKgooHCw8Nr79m9HqTrkrYybNZmatg9PD): protocol: expected piece hash; context canceled; EOF
Aug 9 08:28:14.191655070 (15h2m36.150752305s) [noise=true , address=104.36.82.58:4289 , piece_id=EHP7SPBB2346TPWTDJQXGHGGUKXQFKC4QANJFQQUQQOR47NO3IPQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v4QJpSy7hk4pShMu3FswnZAf3rzyCNUx5sUZSPSvxpecY8T9X): protocol: expected piece hash; context canceled; EOF
Aug 9 08:28:14.191766681 (15h2m36.150863917s) [noise=true , address=64.49.13.169:4706 , piece_id=S2YHCTQ5JCTOYDF7C57HZV74K3GFVBFDKP3GUXNBGZOFY3V7XB2Q, seg_pos={0 0}, upload=15638]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 139.59.144.22:57046->64.49.13.169:4706: use of closed network connection; write tcp 139.59.144.22:57046->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57046->64.49.13.169:4706: use of closed network connection
Aug 9 08:28:14.191834982 (15h2m36.150932218s) [noise=true , address=64.49.13.26:4946 , piece_id=J4G6XFNX6ZIGBMHZF2Z3N37SD3HTTGB6FWZOJOVGJIC2PDDTJSKQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hdE48UnygY1dCRASR39FkumY2V8f5Xb68Pd1WJ4AMLUnnGAfT): protocol: write tcp 139.59.144.22:39096->64.49.13.26:4946: use of closed network connection; write tcp 139.59.144.22:39096->64.49.13.26:4946: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39096->64.49.13.26:4946: use of closed network connection
Aug 9 08:28:14.191848663 (15h2m36.1509459s) [seg_pos={0 0}, upload=15638]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:28:14.191916508 (15h2m36.151013757s) [upload=15638]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:28:14.191921239 (15h2m36.151018488s) [upload=15638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:28:15.172074926 (15h2m37.131172172s) [upload=15638]: Done uploading segments...
Aug 9 08:28:15.172095781 (15h2m37.131193026s) [upload=15638]: Upload finished. err: <nil>
Duration: 2.335378332s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...............X.....................................X......] duration:
860.201256ms
FAIL (2025-08-09 08:47:42 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 860.226343ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [................X.....................................X.....] duration:
840.475808ms
FAIL (2025-08-09 08:46:54 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 840.506594ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [X..................................X........................] duration:
2.339204876s
FAIL (2025-08-09 08:46:42 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
Aug 9 08:28:13.240669419 (15h2m35.199766665s) [upload=15638]: Starting upload
Aug 9 08:28:13.240733944 (15h2m35.199831192s) [upload=15638]: Done starting upload
Aug 9 08:28:13.240872982 (15h2m35.199970234s) [upload=15638]: Uploading segments...
Aug 9 08:28:13.241079499 (15h2m35.200176745s) [upload=15638]: Got next segment. Inline: false
Aug 9 08:28:13.241124567 (15h2m35.200221826s) [upload=15638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 08:28:13.241161881 (15h2m35.200259139s) [upload=15638]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 08:28:13.438488388 (15h2m35.397585691s) [seg_pos={0 0}, upload=15638]: Begin upload segment...
Aug 9 08:28:13.438557720 (15h2m35.397654968s) [seg_pos={0 0}, upload=15638]: Done begin upload segment.
Aug 9 08:28:13.438585890 (15h2m35.397683167s) [noise=true , address=104.36.82.60:4345 , piece_id=WTUSWXUTY4KLXVULDLGKIYL4L6VYL2W6P3HB7W3L3KJIKY6N2NJA, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.438633294 (15h2m35.397730541s) [noise=true , address=77.90.31.19:4872 , piece_id=DU3DZDYSFZG2UQ57QZTYMUZDUCAA443KON37AHMQWET2NJYQWZ6A, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.439039750 (15h2m35.398137001s) [noise=true , address=64.49.13.169:4706 , piece_id=S2YHCTQ5JCTOYDF7C57HZV74K3GFVBFDKP3GUXNBGZOFY3V7XB2Q, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.439054845 (15h2m35.39815209s) [noise=true , address=104.36.82.13:4218 , piece_id=LY2S2YJRPIOVU2NB3JXF7UUX6T4KSVYAJMYVJN5YPAG4FC5BBCFQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.439326357 (15h2m35.398423595s) [noise=true , address=104.36.82.58:4289 , piece_id=EHP7SPBB2346TPWTDJQXGHGGUKXQFKC4QANJFQQUQQOR47NO3IPQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.439434877 (15h2m35.398532129s) [noise=true , address=104.36.82.45:4925 , piece_id=CKPIOOGSRLW6QBQDM2OLQUPNE2BT5YC4EMMDCL3WLTLIMY7SGGZQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.440101858 (15h2m35.399199139s) [noise=true , address=104.36.82.35:4610 , piece_id=KACQNESKG677EBPXOHRRXDAP56EHWV35HKTZQZEY72F4N2T7TS2Q, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.440525550 (15h2m35.399622784s) [noise=true , address=104.36.82.3:4795 , piece_id=Z6E2LED6BNUPDHH4DWHI3MQTYI64XHTMY7ANDSVXSFMGKUS2YPRQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.441368342 (15h2m35.40046559s) [noise=true , address=104.36.82.61:4375 , piece_id=L5YP7MNLUJK7LVW27HUXFKQW4CDNYV6OT44YRJC2LBSCZOQFIT4A, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.441869558 (15h2m35.400966813s) [noise=true , address=104.36.82.18:4385 , piece_id=HXYMSF7VFKZLQ7EDVCBMRNIWWEHJS3JGVRVDJXQNIPVFIPZWH7VQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.442227603 (15h2m35.40132485s) [noise=true , address=104.36.82.23:4470 , piece_id=XTXQA64XHMNOQGOPSYBOHHYKPNXAM2QSNC6ZXFIBOOZOWYI7HFWQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.442628619 (15h2m35.401725871s) [noise=true , address=104.36.82.46:4503 , piece_id=4ZBFEDGEHFA7K76WDNDYT5YMHPLYGP7W4YIYAPV6T7GWPZDKBD7Q, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.442914820 (15h2m35.402012073s) [noise=true , address=64.49.13.26:4946 , piece_id=J4G6XFNX6ZIGBMHZF2Z3N37SD3HTTGB6FWZOJOVGJIC2PDDTJSKQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.443213885 (15h2m35.402311131s) [noise=true , address=104.36.82.54:4176 , piece_id=2G2LW6WR2I5B5ERD3TJQL5WS5TBTPHS6S3O2JZY4USOBXEC53C6A, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.443428267 (15h2m35.402525513s) [noise=true , address=104.36.82.62:4412 , piece_id=ADYVUQ57QEEDZ2FDGPR3C2EKHI4RWOE4U6VRKDHCF3IC3QR2LK2A, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.443813452 (15h2m35.402910703s) [noise=true , address=104.36.82.15:4291 , piece_id=LRAYTCPUN6APCBAE4U4TRHHSPTNDCDUPWK2CF64FOKTKLUDYNWNQ, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.444139434 (15h2m35.403236681s) [noise=true , address=104.36.82.6:4891 , piece_id=7HPMVRYVJXMRDAZSY73CT2GK3YM54BYLVCQ6NRIH7C7U7K7OXEQA, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:13.444576027 (15h2m35.403673276s) [noise=true , address=104.36.82.17:4345 , piece_id=UNNJ3JD4VDZ74N52C5G6QTVJFG2JONR25XTR32F6ZZFZDUH34QEA, seg_pos={0 0}, upload=15638]: Uploading piece...
Aug 9 08:28:14.026403017 (15h2m35.985500265s) [upload=15638]: Next returned nil segment
Aug 9 08:28:14.026499168 (15h2m35.985596416s) [upload=15638]: Waiting for error group managing segments...
Aug 9 08:28:14.178012028 (15h2m36.137109273s) [noise=true , address=77.90.31.19:4872 , piece_id=DU3DZDYSFZG2UQ57QZTYMUZDUCAA443KON37AHMQWET2NJYQWZ6A, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.182219420 (15h2m36.141316665s) [noise=true , address=104.36.82.62:4412 , piece_id=ADYVUQ57QEEDZ2FDGPR3C2EKHI4RWOE4U6VRKDHCF3IC3QR2LK2A, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.182901333 (15h2m36.141998578s) [noise=true , address=104.36.82.13:4218 , piece_id=LY2S2YJRPIOVU2NB3JXF7UUX6T4KSVYAJMYVJN5YPAG4FC5BBCFQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.183653520 (15h2m36.142750765s) [noise=true , address=104.36.82.6:4891 , piece_id=7HPMVRYVJXMRDAZSY73CT2GK3YM54BYLVCQ6NRIH7C7U7K7OXEQA, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.183712240 (15h2m36.142809486s) [noise=true , address=104.36.82.46:4503 , piece_id=4ZBFEDGEHFA7K76WDNDYT5YMHPLYGP7W4YIYAPV6T7GWPZDKBD7Q, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.184021408 (15h2m36.143118654s) [noise=true , address=104.36.82.23:4470 , piece_id=XTXQA64XHMNOQGOPSYBOHHYKPNXAM2QSNC6ZXFIBOOZOWYI7HFWQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.184177333 (15h2m36.143274578s) [noise=true , address=104.36.82.45:4925 , piece_id=CKPIOOGSRLW6QBQDM2OLQUPNE2BT5YC4EMMDCL3WLTLIMY7SGGZQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.184556211 (15h2m36.143653457s) [noise=true , address=104.36.82.3:4795 , piece_id=Z6E2LED6BNUPDHH4DWHI3MQTYI64XHTMY7ANDSVXSFMGKUS2YPRQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.184608088 (15h2m36.143705333s) [noise=true , address=104.36.82.35:4610 , piece_id=KACQNESKG677EBPXOHRRXDAP56EHWV35HKTZQZEY72F4N2T7TS2Q, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.186672666 (15h2m36.145769911s) [noise=true , address=104.36.82.18:4385 , piece_id=HXYMSF7VFKZLQ7EDVCBMRNIWWEHJS3JGVRVDJXQNIPVFIPZWH7VQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.188271372 (15h2m36.147368618s) [noise=true , address=104.36.82.60:4345 , piece_id=WTUSWXUTY4KLXVULDLGKIYL4L6VYL2W6P3HB7W3L3KJIKY6N2NJA, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.190035139 (15h2m36.149132384s) [noise=true , address=104.36.82.61:4375 , piece_id=L5YP7MNLUJK7LVW27HUXFKQW4CDNYV6OT44YRJC2LBSCZOQFIT4A, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.190829073 (15h2m36.149926319s) [noise=true , address=104.36.82.17:4345 , piece_id=UNNJ3JD4VDZ74N52C5G6QTVJFG2JONR25XTR32F6ZZFZDUH34QEA, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.191271905 (15h2m36.150369141s) [noise=true , address=104.36.82.15:4291 , piece_id=LRAYTCPUN6APCBAE4U4TRHHSPTNDCDUPWK2CF64FOKTKLUDYNWNQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: <nil>
Aug 9 08:28:14.191281775 (15h2m36.150379011s) [seg_pos={0 0}, upload=15638]: Segment reached optimal threshold of 14 pieces.
Aug 9 08:28:14.191581487 (15h2m36.150678723s) [noise=true , address=104.36.82.54:4176 , piece_id=2G2LW6WR2I5B5ERD3TJQL5WS5TBTPHS6S3O2JZY4USOBXEC53C6A, seg_pos={0 0}, upload=15638]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Yi5AibNiCNV8vUb9MKgooHCw8Nr79m9HqTrkrYybNZmatg9PD): protocol: expected piece hash; context canceled; EOF
Aug 9 08:28:14.191655070 (15h2m36.150752305s) [noise=true , address=104.36.82.58:4289 , piece_id=EHP7SPBB2346TPWTDJQXGHGGUKXQFKC4QANJFQQUQQOR47NO3IPQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v4QJpSy7hk4pShMu3FswnZAf3rzyCNUx5sUZSPSvxpecY8T9X): protocol: expected piece hash; context canceled; EOF
Aug 9 08:28:14.191766681 (15h2m36.150863917s) [noise=true , address=64.49.13.169:4706 , piece_id=S2YHCTQ5JCTOYDF7C57HZV74K3GFVBFDKP3GUXNBGZOFY3V7XB2Q, seg_pos={0 0}, upload=15638]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 139.59.144.22:57046->64.49.13.169:4706: use of closed network connection; write tcp 139.59.144.22:57046->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57046->64.49.13.169:4706: use of closed network connection
Aug 9 08:28:14.191834982 (15h2m36.150932218s) [noise=true , address=64.49.13.26:4946 , piece_id=J4G6XFNX6ZIGBMHZF2Z3N37SD3HTTGB6FWZOJOVGJIC2PDDTJSKQ, seg_pos={0 0}, upload=15638]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hdE48UnygY1dCRASR39FkumY2V8f5Xb68Pd1WJ4AMLUnnGAfT): protocol: write tcp 139.59.144.22:39096->64.49.13.26:4946: use of closed network connection; write tcp 139.59.144.22:39096->64.49.13.26:4946: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39096->64.49.13.26:4946: use of closed network connection
Aug 9 08:28:14.191848663 (15h2m36.1509459s) [seg_pos={0 0}, upload=15638]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 08:28:14.191916508 (15h2m36.151013757s) [upload=15638]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 08:28:14.191921239 (15h2m36.151018488s) [upload=15638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 08:28:15.172074926 (15h2m37.131172172s) [upload=15638]: Done uploading segments...
Aug 9 08:28:15.172095781 (15h2m37.131193026s) [upload=15638]: Upload finished. err: <nil>
Duration: 2.339242805s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.................X.....................................X....] duration:
866.342437ms
FAIL (2025-08-09 08:45:57 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 08:41:45.094986515 (15h16m7.054083761s) [upload=15880]: Starting upload Aug 9 08:41:45.095037538 (15h16m7.054134783s) [upload=15880]: Done starting upload Aug 9 08:41:45.095117159 (15h16m7.054214404s) [upload=15880]: Uploading segments... Aug 9 08:41:45.095175338 (15h16m7.054272593s) [upload=15880]: Got next segment. Inline: true Aug 9 08:41:45.095180931 (15h16m7.054278179s) [upload=15880]: Waiting for error group managing segments... Aug 9 08:41:45.095234207 (15h16m7.054331453s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 08:41:45.095241903 (15h16m7.054339149s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 08:41:45.095245686 (15h16m7.054342932s) [upload=15880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 08:41:45.263886833 (15h16m7.222984079s) [upload=15880]: Done uploading segments... Aug 9 08:41:45.263905737 (15h16m7.223002983s) [upload=15880]: Upload finished. err: metaclient: Too Many Requests
Duration: 866.379508ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|