Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
1.046841105s
FAIL (2025-09-22 15:04:41 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.046862512s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
2.767041551s
FAIL (2025-09-22 15:04:41 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.767071556s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
2.484342759s
FAIL (2025-09-22 15:04:06 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.484388141s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
2.44393715s
FAIL (2025-09-22 15:03:46 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.443968718s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
2.768468946s
FAIL (2025-09-22 15:03:44 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.768499987s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
1.312962124s
FAIL (2025-09-22 15:03:35 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.312993247s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
2.494726312s
FAIL (2025-09-22 15:03:13 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.494754374s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
2.83677762s
FAIL (2025-09-22 15:03:04 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.836814184s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
1.100076529s
FAIL (2025-09-22 15:02:46 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.100113038s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
2.340811019s
FAIL (2025-09-22 15:02:05 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.340906543s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
2.440860815s
FAIL (2025-09-22 15:02:02 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.440892697s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
1.063601927s
FAIL (2025-09-22 15:01:47 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.063631153s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
2.341280373s
FAIL (2025-09-22 15:01:43 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.341313626s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
2.3942031s
FAIL (2025-09-22 15:01:16 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.394235329s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
2.424812115s
FAIL (2025-09-22 15:00:56 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.424840066s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
2.611817394s
FAIL (2025-09-22 15:00:44 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.611853081s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
1.092629067s
FAIL (2025-09-22 15:00:35 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.092657193s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
1.066531776s
FAIL (2025-09-22 15:00:17 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.066563641s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
2.51534907s
FAIL (2025-09-22 15:00:12 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.515377169s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
2.673912085s
FAIL (2025-09-22 15:00:08 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.673952522s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
1.311345557s
FAIL (2025-09-22 14:59:20 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.311381826s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
2.841077356s
FAIL (2025-09-22 14:59:11 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.841110817s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
2.568325097s
FAIL (2025-09-22 14:59:07 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.568359606s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
2.613762022s
FAIL (2025-09-22 14:58:37 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.613799539s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
1.187588684s
FAIL (2025-09-22 14:58:19 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.187617612s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
2.388624877s
FAIL (2025-09-22 14:58:16 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.388656258s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
2.466733244s
FAIL (2025-09-22 14:58:11 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.466779799s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
1.103347794s
FAIL (2025-09-22 14:57:28 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.103379245s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
2.548322049s
FAIL (2025-09-22 14:57:05 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.548422631s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
2.633058321s
FAIL (2025-09-22 14:57:01 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.63309413s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
2.635339387s
FAIL (2025-09-22 14:56:17 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.635385936s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
1.152904792s
FAIL (2025-09-22 14:56:16 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.152937294s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
2.490967418s
FAIL (2025-09-22 14:55:57 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.490995155s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
2.492346764s
FAIL (2025-09-22 14:55:24 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.492385477s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
1.083874067s
FAIL (2025-09-22 14:55:23 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.083924871s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
2.1441805s
FAIL (2025-09-22 14:55:19 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.144210926s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
2.380579626s
FAIL (2025-09-22 14:54:57 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.380602543s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
1.200979889s
FAIL (2025-09-22 14:54:55 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.20100415s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
2.546110321s
FAIL (2025-09-22 14:54:25 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.546138779s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
1.088072664s
FAIL (2025-09-22 14:54:17 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.088156591s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
2.55946554s
FAIL (2025-09-22 14:54:10 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.559499904s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
2.506294056s
FAIL (2025-09-22 14:53:28 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.506329237s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
2.394607995s
FAIL (2025-09-22 14:53:08 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.394643178s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
1.012591905s
FAIL (2025-09-22 14:53:07 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.012617823s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
2.495071746s
FAIL (2025-09-22 14:53:02 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.495106092s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
1.191251511s
FAIL (2025-09-22 14:52:49 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.191284167s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
2.431593732s
FAIL (2025-09-22 14:52:41 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.431620265s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
982.791403ms
FAIL (2025-09-22 14:52:32 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 982.822217ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
2.497663526s
FAIL (2025-09-22 14:52:10 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.49771227s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
1.000265813s
FAIL (2025-09-22 14:51:47 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.000295115s
core
uplink
us-select-1
|