Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
2.110114677s
FAIL (2025-09-22 14:51:45 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.110155587s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
2.441457749s
FAIL (2025-09-22 14:51:37 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.441492577s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
2.560877564s
FAIL (2025-09-22 14:51:11 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.560910803s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
2.20179839s
FAIL (2025-09-22 14:51:03 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.201830986s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
1.027756173s
FAIL (2025-09-22 14:50:53 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.027778465s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
2.518425731s
FAIL (2025-09-22 14:50:15 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.518457892s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
1.262829197s
FAIL (2025-09-22 14:50:10 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.262865895s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
2.282944161s
FAIL (2025-09-22 14:49:58 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.282994206s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
2.540708075s
FAIL (2025-09-22 14:49:25 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.540737616s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
2.405780137s
FAIL (2025-09-22 14:49:02 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.405823445s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
2.456316132s
FAIL (2025-09-22 14:48:57 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.456344288s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
1.097607921s
FAIL (2025-09-22 14:48:55 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.097644921s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
2.49308001s
FAIL (2025-09-22 14:48:22 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.493119413s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
2.213148765s
FAIL (2025-09-22 14:48:17 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.213177044s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
1.038678379s
FAIL (2025-09-22 14:47:51 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.03873416s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
2.156861501s
FAIL (2025-09-22 14:47:46 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.156894116s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
2.599295587s
FAIL (2025-09-22 14:47:16 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.599330944s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
2.07354113s
FAIL (2025-09-22 14:47:06 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.073576387s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
951.91443ms
FAIL (2025-09-22 14:47:02 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 951.952169ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
2.452208026s
FAIL (2025-09-22 14:46:49 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.452242628s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
2.496982861s
FAIL (2025-09-22 14:46:10 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.497013683s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
1.040517241s
FAIL (2025-09-22 14:46:06 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.040685767s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
2.711015304s
FAIL (2025-09-22 14:46:04 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.711050206s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
5.280734759s
FAIL (2025-09-22 14:45:48 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 5.280768186s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
2.691207144s
FAIL (2025-09-22 14:45:23 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.691254261s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
1.235031714s
FAIL (2025-09-22 14:45:08 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.235062875s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
4.39386125s
FAIL (2025-09-22 14:44:38 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 4.393893408s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
3.655937782s
FAIL (2025-09-22 14:44:36 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 3.655973641s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
2.221241184s
FAIL (2025-09-22 14:44:27 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 2.221275524s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
1.415216053s
FAIL (2025-09-22 14:44:09 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.415314798s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
2.850734045s
FAIL (2025-09-22 14:43:34 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.850769865s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
4.44933218s
FAIL (2025-09-22 14:43:27 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.449357693s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
4.401126692s
FAIL (2025-09-22 14:43:17 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 4.401156828s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
4.39864484s
FAIL (2025-09-22 14:43:08 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 4.398671826s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
1.463176279s
FAIL (2025-09-22 14:42:44 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.463207331s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
4.791777075s
FAIL (2025-09-22 14:42:42 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.79181029s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
2.589358809s
FAIL (2025-09-22 14:42:29 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.589400429s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
4.771972575s
FAIL (2025-09-22 14:42:17 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.772009188s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
1.778013852s
FAIL (2025-09-22 14:42:08 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.778042921s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
4.329806314s
FAIL (2025-09-22 14:41:57 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.329846496s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
3.255684144s
FAIL (2025-09-22 14:41:45 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 3.255716573s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
2.780721385s
FAIL (2025-09-22 14:41:25 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 2.780761455s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
3.606068908s
FAIL (2025-09-22 14:41:11 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 3.606106403s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
1.467825104s
FAIL (2025-09-22 14:41:04 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.467881606s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
2.450039014s
FAIL (2025-09-22 14:40:28 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.450127009s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
3.069873015s
FAIL (2025-09-22 14:40:18 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 3.069909739s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
2.317098855s
FAIL (2025-09-22 14:39:57 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 2.317127502s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
4.057802186s
FAIL (2025-09-22 14:39:51 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.05782922s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
3.554100504s
FAIL (2025-09-22 14:39:46 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Sep 22 14:38:49.764220559 (576h31m10.256653011s) [upload=507140]: Starting upload
Sep 22 14:38:49.764374508 (576h31m10.256806936s) [upload=507140]: Done starting upload
Sep 22 14:38:49.764538141 (576h31m10.256970577s) [upload=507140]: Uploading segments...
Sep 22 14:38:49.764980105 (576h31m10.257412564s) [upload=507140]: Got next segment. Inline: false
Sep 22 14:38:49.765110822 (576h31m10.257543259s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 22 14:38:49.765118388 (576h31m10.257550835s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 22 14:38:59.076060333 (576h31m19.568492762s) [seg_pos={0 0}, upload=507140]: Begin upload segment...
Sep 22 14:38:59.076128276 (576h31m19.568560704s) [seg_pos={0 0}, upload=507140]: Done begin upload segment.
Sep 22 14:38:59.076152863 (576h31m19.568585305s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076516522 (576h31m19.56894897s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.076632936 (576h31m19.569065388s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077376941 (576h31m19.569809392s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.077422979 (576h31m19.569855427s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078067303 (576h31m19.570499745s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078094590 (576h31m19.570527037s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078590865 (576h31m19.571023317s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.078648181 (576h31m19.571080679s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080013326 (576h31m19.572445774s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080494638 (576h31m19.572927084s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.080850524 (576h31m19.573282969s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.081161089 (576h31m19.573593536s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.082939369 (576h31m19.57537182s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083364089 (576h31m19.575796517s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.083990452 (576h31m19.576422895s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Uploading piece...
Sep 22 14:38:59.688913546 (576h31m20.181345974s) [upload=507140]: Next returned nil segment
Sep 22 14:38:59.688933216 (576h31m20.18136566s) [upload=507140]: Waiting for error group managing segments...
Sep 22 14:38:59.827686069 (576h31m20.320118504s) [noise=true , address=45.91.116.11:4118 , piece_id=WSVYB4PF25DFJXU6RMYJBZXBOPUEBMPZWB7OOPGL555SDQSWUDBA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.829820473 (576h31m20.322252916s) [noise=true , address=45.91.116.4:4939 , piece_id=QUBCJ2E5O4GQXBRRLPJAOOVES4JINPSCPVCIUVZ7PV3JU6P7NPIA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.838521617 (576h31m20.330954052s) [noise=true , address=77.90.31.10:4518 , piece_id=S3HI3LDAGAUKZRNLQL6N7MHXWRVC3IITVNY3NYLFW26T6VI7JD7A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839337896 (576h31m20.331770331s) [noise=true , address=45.91.116.8:4048 , piece_id=MVURTE4OT3E7RE5LAA4HUFFH3FR4NFTLFGY443F4RVUXY3BMPMNQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.839490471 (576h31m20.331922902s) [noise=true , address=45.91.116.7:4009 , piece_id=EQOCCYPTJWC6KL5CDVVFCAE345L7HSMS32XWRZNTAM7OBBNXN25Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.841023438 (576h31m20.333455873s) [noise=true , address=45.91.116.8:4045 , piece_id=7VHW3SLZBGNRZNBPJVBED2C5R2RLJNWS2YYDU4BJQ3HYLW3BOHSQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842622235 (576h31m20.335054671s) [noise=true , address=45.91.116.9:4262 , piece_id=5KOPVFOKKKKSOBQWHJZG6JED6OBHYBUNUESDDPYUUQ7DGIIWFZGQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.842716346 (576h31m20.335148797s) [noise=true , address=45.91.116.11:4119 , piece_id=I3BBWVOIK5WW25M7AJ4V7YG2GSJGJ4OTJHZJ6ORAY7B5NU6PZI6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.843993586 (576h31m20.336426028s) [noise=true , address=45.91.116.9:4261 , piece_id=YS2FD4FQV2VZBTYUF7JGLST6WNYUEHTMIA5UZ5LBJWNA3MM725RQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.852953188 (576h31m20.345385623s) [noise=true , address=45.91.116.13:4154 , piece_id=T56XJPBKPICGG5YGF3NMSJEDMQW763LVTKFVSSW633KMVIAHOA6A, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.856945174 (576h31m20.349377627s) [noise=true , address=45.91.116.6:4975 , piece_id=LMRZR6SF6CW7ZH7HOJSQRR6L4TORTKVPST6GOVWFO2W6LDL7X2MQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.868890395 (576h31m20.361322831s) [noise=true , address=77.90.31.17:4770 , piece_id=R267TSBEQSLE5IRCDVXRDJ3W3FWWWV6CCS3KBTWIWRIZGCM7OMVQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904492558 (576h31m20.396924999s) [noise=true , address=45.91.116.5:4298 , piece_id=FTFD4PW5XUYRYQG6CBYKT4SU7P7VQQ2S5A7ZTVXBMWEHM4DPZKTQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: <nil>
Sep 22 14:38:59.904531796 (576h31m20.39696424s) [seg_pos={0 0}, upload=507140]: Segment reached optimal threshold of 13 pieces.
Sep 22 14:38:59.904710538 (576h31m20.397142967s) [noise=true , address=45.91.116.6:4974 , piece_id=77EZSHYJO4RDLITWYYFIF6RRVHIMTRIMNWUKY2C5SAOQJQODJKMA, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904770286 (576h31m20.397202716s) [noise=true , address=45.91.116.4:4938 , piece_id=5SSHRZ2JTKVQ63QFVRTOANYJOMANIUP5TNRVSSGGTQMQ55HEDD5Q, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Sep 22 14:38:59.904835220 (576h31m20.397267649s) [noise=true , address=45.91.116.5:4299 , piece_id=DMZJIP3IMIZARC6UJENXTY5GCJO66XFNXTA6F45FX33GEFVDFDQQ, seg_pos={0 0}, upload=507140]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33446->45.91.116.5:4299: use of closed network connection
Sep 22 14:38:59.904859227 (576h31m20.397291659s) [seg_pos={0 0}, upload=507140]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 22 14:38:59.904972661 (576h31m20.39740509s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 22 14:38:59.904977048 (576h31m20.397409491s) [upload=507140]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 22 14:39:02.092143987 (576h31m22.584576447s) [upload=507140]: Done uploading segments...
Sep 22 14:39:02.092174414 (576h31m22.584606869s) [upload=507140]: Upload finished. err: <nil>
Duration: 3.554126911s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.00104283s
FAIL (2025-09-22 14:39:26 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 30.001085476s
core
uplink
us1
|