Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
15 out of 60 is failed (threshold=3) [......X...........XX.X..X..X..XX.X...........XXXXX...X......] duration:
44.695206994s
FAIL (2025-07-09 01:04:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 44.695233695s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
19.445857095s
FAIL (2025-07-09 01:04:35 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
3980 (375h26m3.9562776s) [upload=300223]: Starting upload
Jul 9 00:37:04.072968784 (375h26m3.956382412s) [upload=300223]: Done starting upload
Jul 9 00:37:04.073087735 (375h26m3.956501365s) [upload=300223]: Uploading segments...
Jul 9 00:37:04.073377778 (375h26m3.956791396s) [upload=300223]: Got next segment. Inline: false
Jul 9 00:37:04.073430757 (375h26m3.956844388s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:37:04.073437151 (375h26m3.956850769s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:37:04.304692457 (375h26m4.18810608s) [seg_pos={0 0}, upload=300223]: Begin upload segment...
Jul 9 00:37:04.304761975 (375h26m4.188175587s) [seg_pos={0 0}, upload=300223]: Done begin upload segment.
Jul 9 00:37:04.304833908 (375h26m4.18824754s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305365539 (375h26m4.18877917s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305738725 (375h26m4.189152359s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305907885 (375h26m4.189321515s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306301693 (375h26m4.189715324s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306679055 (375h26m4.190092689s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307048455 (375h26m4.190462086s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307412956 (375h26m4.19082659s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.308062976 (375h26m4.191476609s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.309126547 (375h26m4.192540184s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.310676166 (375h26m4.194089804s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311146138 (375h26m4.19455977s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311543981 (375h26m4.194957595s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311932868 (375h26m4.195346498s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312324028 (375h26m4.195737661s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312707561 (375h26m4.196121192s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313124992 (375h26m4.19653862s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313485189 (375h26m4.196898821s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.893020764 (375h26m4.776434382s) [upload=300223]: Next returned nil segment
Jul 9 00:37:04.893957299 (375h26m4.777370932s) [upload=300223]: Waiting for error group managing segments...
Jul 9 00:37:05.037354849 (375h26m4.920768466s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.038444983 (375h26m4.921858597s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.039356590 (375h26m4.922770207s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.047559125 (375h26m4.930972742s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.053248590 (375h26m4.93666221s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.054452827 (375h26m4.937866447s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.889326616 (375h26m5.772740232s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:29.628225928 (375h26m29.511639541s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:30.256578837 (375h26m30.139992453s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:32.331498448 (375h26m32.214912065s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:39.883025841 (375h26m39.766439475s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:51.499841545 (375h26m51.383255165s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:55.982236621 (375h26m55.86565024s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607812826 (375h27m3.49122645s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607853389 (375h27m3.491267022s) [seg_pos={0 0}, upload=300223]: Segment reached optimal threshold of 14 pieces.
Jul 9 00:38:03.608006922 (375h27m3.491420552s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608045600 (375h27m3.491459233s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17LrYeCkrPWdiL9gGZ568PvfFct6cLVRQE4Qa92ykLgqXSfBL9): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608087309 (375h27m3.491500937s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RY6XGL5MJYc2idL6Np7DS3U9pCQGDhuEEqguY8AgjasjE4hiZ): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608191697 (375h27m3.491605324s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T6FqqMunZ46QoxhHVWsn8NinZGWv1Yo7rNHSiK6s9hhuQtS6Y): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608213412 (375h27m3.49162703s) [seg_pos={0 0}, upload=300223]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 00:38:03.608310462 (375h27m3.49172408s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:38:03.608316810 (375h27m3.491730438s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:03.761052568 (375h27m3.6444662s) [upload=300223]: Done uploading segments...
Jul 9 00:38:03.761076757 (375h27m3.644490378s) [upload=300223]: Upload finished. err: <nil>
Duration: 19.445916179s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
14 out of 60 is failed (threshold=3) [...........X....X..X......X.............XX.X.X......X.X.XXXX] duration:
30.001022427s
FAIL (2025-07-09 01:04:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001073284s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
19.114864408s
FAIL (2025-07-09 01:04:08 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
3980 (375h26m3.9562776s) [upload=300223]: Starting upload
Jul 9 00:37:04.072968784 (375h26m3.956382412s) [upload=300223]: Done starting upload
Jul 9 00:37:04.073087735 (375h26m3.956501365s) [upload=300223]: Uploading segments...
Jul 9 00:37:04.073377778 (375h26m3.956791396s) [upload=300223]: Got next segment. Inline: false
Jul 9 00:37:04.073430757 (375h26m3.956844388s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:37:04.073437151 (375h26m3.956850769s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:37:04.304692457 (375h26m4.18810608s) [seg_pos={0 0}, upload=300223]: Begin upload segment...
Jul 9 00:37:04.304761975 (375h26m4.188175587s) [seg_pos={0 0}, upload=300223]: Done begin upload segment.
Jul 9 00:37:04.304833908 (375h26m4.18824754s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305365539 (375h26m4.18877917s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305738725 (375h26m4.189152359s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305907885 (375h26m4.189321515s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306301693 (375h26m4.189715324s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306679055 (375h26m4.190092689s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307048455 (375h26m4.190462086s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307412956 (375h26m4.19082659s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.308062976 (375h26m4.191476609s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.309126547 (375h26m4.192540184s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.310676166 (375h26m4.194089804s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311146138 (375h26m4.19455977s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311543981 (375h26m4.194957595s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311932868 (375h26m4.195346498s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312324028 (375h26m4.195737661s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312707561 (375h26m4.196121192s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313124992 (375h26m4.19653862s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313485189 (375h26m4.196898821s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.893020764 (375h26m4.776434382s) [upload=300223]: Next returned nil segment
Jul 9 00:37:04.893957299 (375h26m4.777370932s) [upload=300223]: Waiting for error group managing segments...
Jul 9 00:37:05.037354849 (375h26m4.920768466s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.038444983 (375h26m4.921858597s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.039356590 (375h26m4.922770207s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.047559125 (375h26m4.930972742s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.053248590 (375h26m4.93666221s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.054452827 (375h26m4.937866447s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.889326616 (375h26m5.772740232s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:29.628225928 (375h26m29.511639541s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:30.256578837 (375h26m30.139992453s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:32.331498448 (375h26m32.214912065s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:39.883025841 (375h26m39.766439475s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:51.499841545 (375h26m51.383255165s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:55.982236621 (375h26m55.86565024s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607812826 (375h27m3.49122645s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607853389 (375h27m3.491267022s) [seg_pos={0 0}, upload=300223]: Segment reached optimal threshold of 14 pieces.
Jul 9 00:38:03.608006922 (375h27m3.491420552s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608045600 (375h27m3.491459233s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17LrYeCkrPWdiL9gGZ568PvfFct6cLVRQE4Qa92ykLgqXSfBL9): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608087309 (375h27m3.491500937s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RY6XGL5MJYc2idL6Np7DS3U9pCQGDhuEEqguY8AgjasjE4hiZ): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608191697 (375h27m3.491605324s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T6FqqMunZ46QoxhHVWsn8NinZGWv1Yo7rNHSiK6s9hhuQtS6Y): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608213412 (375h27m3.49162703s) [seg_pos={0 0}, upload=300223]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 00:38:03.608310462 (375h27m3.49172408s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:38:03.608316810 (375h27m3.491730438s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:03.761052568 (375h27m3.6444662s) [upload=300223]: Done uploading segments...
Jul 9 00:38:03.761076757 (375h27m3.644490378s) [upload=300223]: Upload finished. err: <nil>
Duration: 19.114890138s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......................X..X.................................] duration:
18.617650226s
FAIL (2025-07-09 01:04:06 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 10)
======LOGS======
entCommit
Jul 9 00:38:54.141867066 (375h27m54.025280684s) [upload=300248]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:54.286588243 (375h27m54.170001881s) [upload=300248]: Done uploading segments...
Jul 9 00:38:54.286633475 (375h27m54.170047105s) [upload=300248]: Upload finished. err: <nil>
Jul 9 00:39:49.853458099 (375h28m49.736871729s) [upload=300262]: Starting upload
Jul 9 00:39:49.853564556 (375h28m49.736978174s) [upload=300262]: Done starting upload
Jul 9 00:39:49.853754759 (375h28m49.737168394s) [upload=300262]: Uploading segments...
Jul 9 00:39:49.854072231 (375h28m49.737485857s) [upload=300262]: Got next segment. Inline: false
Jul 9 00:39:49.854139888 (375h28m49.737553515s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:39:49.854145670 (375h28m49.737559297s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:39:49.964717413 (375h28m49.848131039s) [seg_pos={0 0}, upload=300262]: Begin upload segment...
Jul 9 00:39:49.964772224 (375h28m49.848185839s) [seg_pos={0 0}, upload=300262]: Done begin upload segment.
Jul 9 00:39:49.964909767 (375h28m49.848323393s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965422318 (375h28m49.848835947s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965585955 (375h28m49.848999581s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966217091 (375h28m49.849630727s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966292916 (375h28m49.849706542s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967093655 (375h28m49.85050729s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967169721 (375h28m49.850583348s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967940005 (375h28m49.851353634s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.968588971 (375h28m49.8520026s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969080450 (375h28m49.852494085s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969563775 (375h28m49.852977409s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.970736354 (375h28m49.854149991s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.971229734 (375h28m49.85464336s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.972867679 (375h28m49.856281317s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.978267895 (375h28m49.861681529s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.986076032 (375h28m49.869489662s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:50.595435555 (375h28m50.478849173s) [upload=300262]: Next returned nil segment
Jul 9 00:39:50.595455642 (375h28m50.478869272s) [upload=300262]: Waiting for error group managing segments...
Jul 9 00:39:50.730628340 (375h28m50.614041956s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.731179161 (375h28m50.614592788s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.732951876 (375h28m50.616365493s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.738403976 (375h28m50.621817592s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739011956 (375h28m50.622425573s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739658988 (375h28m50.623072606s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.766298764 (375h28m50.649712392s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.754942777 (375h28m51.638356391s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.990915358 (375h28m51.87432899s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:13.196278807 (375h29m13.079692437s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:19.575930850 (375h29m19.459344476s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576005813 (375h29m19.459419439s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576051734 (375h29m19.459465361s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576066765 (375h29m19.459480398s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576255242 (375h29m19.459668869s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection
Jul 9 00:40:19.576293041 (375h29m19.459706667s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection
Jul 9 00:40:19.576316511 (375h29m19.45973014s) [seg_pos={0 0}, upload=300262]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil>
Jul 9 00:40:19.685159343 (375h29m19.568572971s) [upload=300262]: Done uploading segments...
Jul 9 00:40:19.685187481 (375h29m19.568601108s) [upload=300262]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10)
Duration: 18.61769303s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
15 out of 60 is failed (threshold=3) [.......X...........XX.X..X..X..XX.X...........XXXXX...X.....] duration:
51.304680539s
FAIL (2025-07-09 01:03:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 51.304702972s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
33.356310302s
FAIL (2025-07-09 01:03:49 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
3980 (375h26m3.9562776s) [upload=300223]: Starting upload
Jul 9 00:37:04.072968784 (375h26m3.956382412s) [upload=300223]: Done starting upload
Jul 9 00:37:04.073087735 (375h26m3.956501365s) [upload=300223]: Uploading segments...
Jul 9 00:37:04.073377778 (375h26m3.956791396s) [upload=300223]: Got next segment. Inline: false
Jul 9 00:37:04.073430757 (375h26m3.956844388s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:37:04.073437151 (375h26m3.956850769s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:37:04.304692457 (375h26m4.18810608s) [seg_pos={0 0}, upload=300223]: Begin upload segment...
Jul 9 00:37:04.304761975 (375h26m4.188175587s) [seg_pos={0 0}, upload=300223]: Done begin upload segment.
Jul 9 00:37:04.304833908 (375h26m4.18824754s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305365539 (375h26m4.18877917s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305738725 (375h26m4.189152359s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305907885 (375h26m4.189321515s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306301693 (375h26m4.189715324s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306679055 (375h26m4.190092689s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307048455 (375h26m4.190462086s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307412956 (375h26m4.19082659s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.308062976 (375h26m4.191476609s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.309126547 (375h26m4.192540184s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.310676166 (375h26m4.194089804s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311146138 (375h26m4.19455977s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311543981 (375h26m4.194957595s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311932868 (375h26m4.195346498s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312324028 (375h26m4.195737661s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312707561 (375h26m4.196121192s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313124992 (375h26m4.19653862s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313485189 (375h26m4.196898821s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.893020764 (375h26m4.776434382s) [upload=300223]: Next returned nil segment
Jul 9 00:37:04.893957299 (375h26m4.777370932s) [upload=300223]: Waiting for error group managing segments...
Jul 9 00:37:05.037354849 (375h26m4.920768466s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.038444983 (375h26m4.921858597s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.039356590 (375h26m4.922770207s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.047559125 (375h26m4.930972742s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.053248590 (375h26m4.93666221s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.054452827 (375h26m4.937866447s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.889326616 (375h26m5.772740232s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:29.628225928 (375h26m29.511639541s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:30.256578837 (375h26m30.139992453s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:32.331498448 (375h26m32.214912065s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:39.883025841 (375h26m39.766439475s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:51.499841545 (375h26m51.383255165s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:55.982236621 (375h26m55.86565024s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607812826 (375h27m3.49122645s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607853389 (375h27m3.491267022s) [seg_pos={0 0}, upload=300223]: Segment reached optimal threshold of 14 pieces.
Jul 9 00:38:03.608006922 (375h27m3.491420552s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608045600 (375h27m3.491459233s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17LrYeCkrPWdiL9gGZ568PvfFct6cLVRQE4Qa92ykLgqXSfBL9): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608087309 (375h27m3.491500937s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RY6XGL5MJYc2idL6Np7DS3U9pCQGDhuEEqguY8AgjasjE4hiZ): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608191697 (375h27m3.491605324s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T6FqqMunZ46QoxhHVWsn8NinZGWv1Yo7rNHSiK6s9hhuQtS6Y): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608213412 (375h27m3.49162703s) [seg_pos={0 0}, upload=300223]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 00:38:03.608310462 (375h27m3.49172408s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:38:03.608316810 (375h27m3.491730438s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:03.761052568 (375h27m3.6444662s) [upload=300223]: Done uploading segments...
Jul 9 00:38:03.761076757 (375h27m3.644490378s) [upload=300223]: Upload finished. err: <nil>
Duration: 33.356343695s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [............X....X..X......X.............XX.X.X......X.X.XXX] duration:
30.001171271s
FAIL (2025-07-09 01:03:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001220181s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [........................X..X................................] duration:
10.248953061s
FAIL (2025-07-09 01:03:12 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 10)
======LOGS======
entCommit
Jul 9 00:38:54.141867066 (375h27m54.025280684s) [upload=300248]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:54.286588243 (375h27m54.170001881s) [upload=300248]: Done uploading segments...
Jul 9 00:38:54.286633475 (375h27m54.170047105s) [upload=300248]: Upload finished. err: <nil>
Jul 9 00:39:49.853458099 (375h28m49.736871729s) [upload=300262]: Starting upload
Jul 9 00:39:49.853564556 (375h28m49.736978174s) [upload=300262]: Done starting upload
Jul 9 00:39:49.853754759 (375h28m49.737168394s) [upload=300262]: Uploading segments...
Jul 9 00:39:49.854072231 (375h28m49.737485857s) [upload=300262]: Got next segment. Inline: false
Jul 9 00:39:49.854139888 (375h28m49.737553515s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:39:49.854145670 (375h28m49.737559297s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:39:49.964717413 (375h28m49.848131039s) [seg_pos={0 0}, upload=300262]: Begin upload segment...
Jul 9 00:39:49.964772224 (375h28m49.848185839s) [seg_pos={0 0}, upload=300262]: Done begin upload segment.
Jul 9 00:39:49.964909767 (375h28m49.848323393s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965422318 (375h28m49.848835947s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965585955 (375h28m49.848999581s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966217091 (375h28m49.849630727s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966292916 (375h28m49.849706542s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967093655 (375h28m49.85050729s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967169721 (375h28m49.850583348s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967940005 (375h28m49.851353634s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.968588971 (375h28m49.8520026s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969080450 (375h28m49.852494085s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969563775 (375h28m49.852977409s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.970736354 (375h28m49.854149991s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.971229734 (375h28m49.85464336s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.972867679 (375h28m49.856281317s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.978267895 (375h28m49.861681529s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.986076032 (375h28m49.869489662s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:50.595435555 (375h28m50.478849173s) [upload=300262]: Next returned nil segment
Jul 9 00:39:50.595455642 (375h28m50.478869272s) [upload=300262]: Waiting for error group managing segments...
Jul 9 00:39:50.730628340 (375h28m50.614041956s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.731179161 (375h28m50.614592788s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.732951876 (375h28m50.616365493s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.738403976 (375h28m50.621817592s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739011956 (375h28m50.622425573s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739658988 (375h28m50.623072606s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.766298764 (375h28m50.649712392s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.754942777 (375h28m51.638356391s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.990915358 (375h28m51.87432899s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:13.196278807 (375h29m13.079692437s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:19.575930850 (375h29m19.459344476s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576005813 (375h29m19.459419439s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576051734 (375h29m19.459465361s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576066765 (375h29m19.459480398s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576255242 (375h29m19.459668869s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection
Jul 9 00:40:19.576293041 (375h29m19.459706667s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection
Jul 9 00:40:19.576316511 (375h29m19.45973014s) [seg_pos={0 0}, upload=300262]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil>
Jul 9 00:40:19.685159343 (375h29m19.568572971s) [upload=300262]: Done uploading segments...
Jul 9 00:40:19.685187481 (375h29m19.568601108s) [upload=300262]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10)
Duration: 10.248988026s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
15 out of 60 is failed (threshold=3) [........X...........XX.X..X..X..XX.X...........XXXXX...X....] duration:
42.299609455s
FAIL (2025-07-09 01:03:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 42.29971019s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
12 out of 60 is failed (threshold=3) [.............X....X..X......X.............XX.X.X......X.X.XX] duration:
30.000685044s
FAIL (2025-07-09 01:03:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000712992s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
24.489508911s
FAIL (2025-07-09 01:02:53 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
3980 (375h26m3.9562776s) [upload=300223]: Starting upload
Jul 9 00:37:04.072968784 (375h26m3.956382412s) [upload=300223]: Done starting upload
Jul 9 00:37:04.073087735 (375h26m3.956501365s) [upload=300223]: Uploading segments...
Jul 9 00:37:04.073377778 (375h26m3.956791396s) [upload=300223]: Got next segment. Inline: false
Jul 9 00:37:04.073430757 (375h26m3.956844388s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:37:04.073437151 (375h26m3.956850769s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:37:04.304692457 (375h26m4.18810608s) [seg_pos={0 0}, upload=300223]: Begin upload segment...
Jul 9 00:37:04.304761975 (375h26m4.188175587s) [seg_pos={0 0}, upload=300223]: Done begin upload segment.
Jul 9 00:37:04.304833908 (375h26m4.18824754s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305365539 (375h26m4.18877917s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305738725 (375h26m4.189152359s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305907885 (375h26m4.189321515s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306301693 (375h26m4.189715324s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306679055 (375h26m4.190092689s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307048455 (375h26m4.190462086s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307412956 (375h26m4.19082659s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.308062976 (375h26m4.191476609s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.309126547 (375h26m4.192540184s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.310676166 (375h26m4.194089804s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311146138 (375h26m4.19455977s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311543981 (375h26m4.194957595s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311932868 (375h26m4.195346498s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312324028 (375h26m4.195737661s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312707561 (375h26m4.196121192s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313124992 (375h26m4.19653862s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313485189 (375h26m4.196898821s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.893020764 (375h26m4.776434382s) [upload=300223]: Next returned nil segment
Jul 9 00:37:04.893957299 (375h26m4.777370932s) [upload=300223]: Waiting for error group managing segments...
Jul 9 00:37:05.037354849 (375h26m4.920768466s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.038444983 (375h26m4.921858597s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.039356590 (375h26m4.922770207s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.047559125 (375h26m4.930972742s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.053248590 (375h26m4.93666221s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.054452827 (375h26m4.937866447s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.889326616 (375h26m5.772740232s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:29.628225928 (375h26m29.511639541s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:30.256578837 (375h26m30.139992453s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:32.331498448 (375h26m32.214912065s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:39.883025841 (375h26m39.766439475s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:51.499841545 (375h26m51.383255165s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:55.982236621 (375h26m55.86565024s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607812826 (375h27m3.49122645s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607853389 (375h27m3.491267022s) [seg_pos={0 0}, upload=300223]: Segment reached optimal threshold of 14 pieces.
Jul 9 00:38:03.608006922 (375h27m3.491420552s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608045600 (375h27m3.491459233s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17LrYeCkrPWdiL9gGZ568PvfFct6cLVRQE4Qa92ykLgqXSfBL9): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608087309 (375h27m3.491500937s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RY6XGL5MJYc2idL6Np7DS3U9pCQGDhuEEqguY8AgjasjE4hiZ): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608191697 (375h27m3.491605324s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T6FqqMunZ46QoxhHVWsn8NinZGWv1Yo7rNHSiK6s9hhuQtS6Y): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608213412 (375h27m3.49162703s) [seg_pos={0 0}, upload=300223]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 00:38:03.608310462 (375h27m3.49172408s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:38:03.608316810 (375h27m3.491730438s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:03.761052568 (375h27m3.6444662s) [upload=300223]: Done uploading segments...
Jul 9 00:38:03.761076757 (375h27m3.644490378s) [upload=300223]: Upload finished. err: <nil>
Duration: 24.489546897s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........................X..X...............................] duration:
17.397313876s
FAIL (2025-07-09 01:02:31 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 10)
======LOGS======
entCommit
Jul 9 00:38:54.141867066 (375h27m54.025280684s) [upload=300248]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:54.286588243 (375h27m54.170001881s) [upload=300248]: Done uploading segments...
Jul 9 00:38:54.286633475 (375h27m54.170047105s) [upload=300248]: Upload finished. err: <nil>
Jul 9 00:39:49.853458099 (375h28m49.736871729s) [upload=300262]: Starting upload
Jul 9 00:39:49.853564556 (375h28m49.736978174s) [upload=300262]: Done starting upload
Jul 9 00:39:49.853754759 (375h28m49.737168394s) [upload=300262]: Uploading segments...
Jul 9 00:39:49.854072231 (375h28m49.737485857s) [upload=300262]: Got next segment. Inline: false
Jul 9 00:39:49.854139888 (375h28m49.737553515s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:39:49.854145670 (375h28m49.737559297s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:39:49.964717413 (375h28m49.848131039s) [seg_pos={0 0}, upload=300262]: Begin upload segment...
Jul 9 00:39:49.964772224 (375h28m49.848185839s) [seg_pos={0 0}, upload=300262]: Done begin upload segment.
Jul 9 00:39:49.964909767 (375h28m49.848323393s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965422318 (375h28m49.848835947s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965585955 (375h28m49.848999581s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966217091 (375h28m49.849630727s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966292916 (375h28m49.849706542s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967093655 (375h28m49.85050729s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967169721 (375h28m49.850583348s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967940005 (375h28m49.851353634s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.968588971 (375h28m49.8520026s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969080450 (375h28m49.852494085s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969563775 (375h28m49.852977409s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.970736354 (375h28m49.854149991s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.971229734 (375h28m49.85464336s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.972867679 (375h28m49.856281317s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.978267895 (375h28m49.861681529s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.986076032 (375h28m49.869489662s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:50.595435555 (375h28m50.478849173s) [upload=300262]: Next returned nil segment
Jul 9 00:39:50.595455642 (375h28m50.478869272s) [upload=300262]: Waiting for error group managing segments...
Jul 9 00:39:50.730628340 (375h28m50.614041956s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.731179161 (375h28m50.614592788s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.732951876 (375h28m50.616365493s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.738403976 (375h28m50.621817592s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739011956 (375h28m50.622425573s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739658988 (375h28m50.623072606s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.766298764 (375h28m50.649712392s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.754942777 (375h28m51.638356391s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.990915358 (375h28m51.87432899s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:13.196278807 (375h29m13.079692437s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:19.575930850 (375h29m19.459344476s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576005813 (375h29m19.459419439s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576051734 (375h29m19.459465361s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576066765 (375h29m19.459480398s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576255242 (375h29m19.459668869s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection
Jul 9 00:40:19.576293041 (375h29m19.459706667s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection
Jul 9 00:40:19.576316511 (375h29m19.45973014s) [seg_pos={0 0}, upload=300262]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil>
Jul 9 00:40:19.685159343 (375h29m19.568572971s) [upload=300262]: Done uploading segments...
Jul 9 00:40:19.685187481 (375h29m19.568601108s) [upload=300262]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10)
Duration: 17.397369194s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
15 out of 60 is failed (threshold=3) [.........X...........XX.X..X..X..XX.X...........XXXXX...X...] duration:
41.182333381s
FAIL (2025-07-09 01:02:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 41.182369778s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
11 out of 60 is failed (threshold=3) [..............X....X..X......X.............XX.X.X......X.X.X] duration:
30.000994573s
FAIL (2025-07-09 01:02:11 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001026491s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
20.195621823s
FAIL (2025-07-09 01:01:57 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
3980 (375h26m3.9562776s) [upload=300223]: Starting upload
Jul 9 00:37:04.072968784 (375h26m3.956382412s) [upload=300223]: Done starting upload
Jul 9 00:37:04.073087735 (375h26m3.956501365s) [upload=300223]: Uploading segments...
Jul 9 00:37:04.073377778 (375h26m3.956791396s) [upload=300223]: Got next segment. Inline: false
Jul 9 00:37:04.073430757 (375h26m3.956844388s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:37:04.073437151 (375h26m3.956850769s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:37:04.304692457 (375h26m4.18810608s) [seg_pos={0 0}, upload=300223]: Begin upload segment...
Jul 9 00:37:04.304761975 (375h26m4.188175587s) [seg_pos={0 0}, upload=300223]: Done begin upload segment.
Jul 9 00:37:04.304833908 (375h26m4.18824754s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305365539 (375h26m4.18877917s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305738725 (375h26m4.189152359s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305907885 (375h26m4.189321515s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306301693 (375h26m4.189715324s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306679055 (375h26m4.190092689s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307048455 (375h26m4.190462086s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307412956 (375h26m4.19082659s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.308062976 (375h26m4.191476609s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.309126547 (375h26m4.192540184s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.310676166 (375h26m4.194089804s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311146138 (375h26m4.19455977s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311543981 (375h26m4.194957595s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311932868 (375h26m4.195346498s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312324028 (375h26m4.195737661s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312707561 (375h26m4.196121192s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313124992 (375h26m4.19653862s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313485189 (375h26m4.196898821s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.893020764 (375h26m4.776434382s) [upload=300223]: Next returned nil segment
Jul 9 00:37:04.893957299 (375h26m4.777370932s) [upload=300223]: Waiting for error group managing segments...
Jul 9 00:37:05.037354849 (375h26m4.920768466s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.038444983 (375h26m4.921858597s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.039356590 (375h26m4.922770207s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.047559125 (375h26m4.930972742s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.053248590 (375h26m4.93666221s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.054452827 (375h26m4.937866447s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.889326616 (375h26m5.772740232s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:29.628225928 (375h26m29.511639541s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:30.256578837 (375h26m30.139992453s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:32.331498448 (375h26m32.214912065s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:39.883025841 (375h26m39.766439475s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:51.499841545 (375h26m51.383255165s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:55.982236621 (375h26m55.86565024s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607812826 (375h27m3.49122645s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607853389 (375h27m3.491267022s) [seg_pos={0 0}, upload=300223]: Segment reached optimal threshold of 14 pieces.
Jul 9 00:38:03.608006922 (375h27m3.491420552s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608045600 (375h27m3.491459233s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17LrYeCkrPWdiL9gGZ568PvfFct6cLVRQE4Qa92ykLgqXSfBL9): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608087309 (375h27m3.491500937s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RY6XGL5MJYc2idL6Np7DS3U9pCQGDhuEEqguY8AgjasjE4hiZ): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608191697 (375h27m3.491605324s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T6FqqMunZ46QoxhHVWsn8NinZGWv1Yo7rNHSiK6s9hhuQtS6Y): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608213412 (375h27m3.49162703s) [seg_pos={0 0}, upload=300223]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 00:38:03.608310462 (375h27m3.49172408s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:38:03.608316810 (375h27m3.491730438s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:03.761052568 (375h27m3.6444662s) [upload=300223]: Done uploading segments...
Jul 9 00:38:03.761076757 (375h27m3.644490378s) [upload=300223]: Upload finished. err: <nil>
Duration: 20.195658031s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................X..X..............................] duration:
2.884998619s
FAIL (2025-07-09 01:01:52 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 10)
======LOGS======
entCommit
Jul 9 00:38:54.141867066 (375h27m54.025280684s) [upload=300248]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:54.286588243 (375h27m54.170001881s) [upload=300248]: Done uploading segments...
Jul 9 00:38:54.286633475 (375h27m54.170047105s) [upload=300248]: Upload finished. err: <nil>
Jul 9 00:39:49.853458099 (375h28m49.736871729s) [upload=300262]: Starting upload
Jul 9 00:39:49.853564556 (375h28m49.736978174s) [upload=300262]: Done starting upload
Jul 9 00:39:49.853754759 (375h28m49.737168394s) [upload=300262]: Uploading segments...
Jul 9 00:39:49.854072231 (375h28m49.737485857s) [upload=300262]: Got next segment. Inline: false
Jul 9 00:39:49.854139888 (375h28m49.737553515s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:39:49.854145670 (375h28m49.737559297s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:39:49.964717413 (375h28m49.848131039s) [seg_pos={0 0}, upload=300262]: Begin upload segment...
Jul 9 00:39:49.964772224 (375h28m49.848185839s) [seg_pos={0 0}, upload=300262]: Done begin upload segment.
Jul 9 00:39:49.964909767 (375h28m49.848323393s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965422318 (375h28m49.848835947s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965585955 (375h28m49.848999581s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966217091 (375h28m49.849630727s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966292916 (375h28m49.849706542s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967093655 (375h28m49.85050729s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967169721 (375h28m49.850583348s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967940005 (375h28m49.851353634s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.968588971 (375h28m49.8520026s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969080450 (375h28m49.852494085s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969563775 (375h28m49.852977409s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.970736354 (375h28m49.854149991s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.971229734 (375h28m49.85464336s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.972867679 (375h28m49.856281317s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.978267895 (375h28m49.861681529s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.986076032 (375h28m49.869489662s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:50.595435555 (375h28m50.478849173s) [upload=300262]: Next returned nil segment
Jul 9 00:39:50.595455642 (375h28m50.478869272s) [upload=300262]: Waiting for error group managing segments...
Jul 9 00:39:50.730628340 (375h28m50.614041956s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.731179161 (375h28m50.614592788s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.732951876 (375h28m50.616365493s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.738403976 (375h28m50.621817592s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739011956 (375h28m50.622425573s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739658988 (375h28m50.623072606s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.766298764 (375h28m50.649712392s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.754942777 (375h28m51.638356391s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.990915358 (375h28m51.87432899s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:13.196278807 (375h29m13.079692437s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:19.575930850 (375h29m19.459344476s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576005813 (375h29m19.459419439s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576051734 (375h29m19.459465361s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576066765 (375h29m19.459480398s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576255242 (375h29m19.459668869s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection
Jul 9 00:40:19.576293041 (375h29m19.459706667s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection
Jul 9 00:40:19.576316511 (375h29m19.45973014s) [seg_pos={0 0}, upload=300262]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil>
Jul 9 00:40:19.685159343 (375h29m19.568572971s) [upload=300262]: Done uploading segments...
Jul 9 00:40:19.685187481 (375h29m19.568601108s) [upload=300262]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10)
Duration: 2.885027254s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
14.179663877s
FAIL (2025-07-09 01:01:22 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
3980 (375h26m3.9562776s) [upload=300223]: Starting upload
Jul 9 00:37:04.072968784 (375h26m3.956382412s) [upload=300223]: Done starting upload
Jul 9 00:37:04.073087735 (375h26m3.956501365s) [upload=300223]: Uploading segments...
Jul 9 00:37:04.073377778 (375h26m3.956791396s) [upload=300223]: Got next segment. Inline: false
Jul 9 00:37:04.073430757 (375h26m3.956844388s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:37:04.073437151 (375h26m3.956850769s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:37:04.304692457 (375h26m4.18810608s) [seg_pos={0 0}, upload=300223]: Begin upload segment...
Jul 9 00:37:04.304761975 (375h26m4.188175587s) [seg_pos={0 0}, upload=300223]: Done begin upload segment.
Jul 9 00:37:04.304833908 (375h26m4.18824754s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305365539 (375h26m4.18877917s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305738725 (375h26m4.189152359s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305907885 (375h26m4.189321515s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306301693 (375h26m4.189715324s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306679055 (375h26m4.190092689s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307048455 (375h26m4.190462086s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307412956 (375h26m4.19082659s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.308062976 (375h26m4.191476609s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.309126547 (375h26m4.192540184s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.310676166 (375h26m4.194089804s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311146138 (375h26m4.19455977s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311543981 (375h26m4.194957595s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311932868 (375h26m4.195346498s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312324028 (375h26m4.195737661s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312707561 (375h26m4.196121192s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313124992 (375h26m4.19653862s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313485189 (375h26m4.196898821s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.893020764 (375h26m4.776434382s) [upload=300223]: Next returned nil segment
Jul 9 00:37:04.893957299 (375h26m4.777370932s) [upload=300223]: Waiting for error group managing segments...
Jul 9 00:37:05.037354849 (375h26m4.920768466s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.038444983 (375h26m4.921858597s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.039356590 (375h26m4.922770207s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.047559125 (375h26m4.930972742s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.053248590 (375h26m4.93666221s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.054452827 (375h26m4.937866447s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.889326616 (375h26m5.772740232s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:29.628225928 (375h26m29.511639541s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:30.256578837 (375h26m30.139992453s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:32.331498448 (375h26m32.214912065s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:39.883025841 (375h26m39.766439475s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:51.499841545 (375h26m51.383255165s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:55.982236621 (375h26m55.86565024s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607812826 (375h27m3.49122645s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607853389 (375h27m3.491267022s) [seg_pos={0 0}, upload=300223]: Segment reached optimal threshold of 14 pieces.
Jul 9 00:38:03.608006922 (375h27m3.491420552s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608045600 (375h27m3.491459233s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17LrYeCkrPWdiL9gGZ568PvfFct6cLVRQE4Qa92ykLgqXSfBL9): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608087309 (375h27m3.491500937s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RY6XGL5MJYc2idL6Np7DS3U9pCQGDhuEEqguY8AgjasjE4hiZ): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608191697 (375h27m3.491605324s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T6FqqMunZ46QoxhHVWsn8NinZGWv1Yo7rNHSiK6s9hhuQtS6Y): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608213412 (375h27m3.49162703s) [seg_pos={0 0}, upload=300223]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 00:38:03.608310462 (375h27m3.49172408s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:38:03.608316810 (375h27m3.491730438s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:03.761052568 (375h27m3.6444662s) [upload=300223]: Done uploading segments...
Jul 9 00:38:03.761076757 (375h27m3.644490378s) [upload=300223]: Upload finished. err: <nil>
Duration: 14.179723323s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
15 out of 60 is failed (threshold=3) [..........X...........XX.X..X..X..XX.X...........XXXXX...X..] duration:
35.591809085s
FAIL (2025-07-09 01:01:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 35.59182413s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
11 out of 60 is failed (threshold=3) [X..............X....X..X......X.............XX.X.X......X.X.] duration:
3.558347011s
FAIL (2025-07-09 01:01:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.558368353s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
11 out of 60 is failed (threshold=3) [.X..............X....X..X......X.............XX.X.X......X.X] duration:
30.001158546s
FAIL (2025-07-09 01:00:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001180086s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
19.66923616s
FAIL (2025-07-09 01:00:49 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
3980 (375h26m3.9562776s) [upload=300223]: Starting upload
Jul 9 00:37:04.072968784 (375h26m3.956382412s) [upload=300223]: Done starting upload
Jul 9 00:37:04.073087735 (375h26m3.956501365s) [upload=300223]: Uploading segments...
Jul 9 00:37:04.073377778 (375h26m3.956791396s) [upload=300223]: Got next segment. Inline: false
Jul 9 00:37:04.073430757 (375h26m3.956844388s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:37:04.073437151 (375h26m3.956850769s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:37:04.304692457 (375h26m4.18810608s) [seg_pos={0 0}, upload=300223]: Begin upload segment...
Jul 9 00:37:04.304761975 (375h26m4.188175587s) [seg_pos={0 0}, upload=300223]: Done begin upload segment.
Jul 9 00:37:04.304833908 (375h26m4.18824754s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305365539 (375h26m4.18877917s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305738725 (375h26m4.189152359s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305907885 (375h26m4.189321515s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306301693 (375h26m4.189715324s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306679055 (375h26m4.190092689s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307048455 (375h26m4.190462086s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307412956 (375h26m4.19082659s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.308062976 (375h26m4.191476609s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.309126547 (375h26m4.192540184s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.310676166 (375h26m4.194089804s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311146138 (375h26m4.19455977s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311543981 (375h26m4.194957595s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311932868 (375h26m4.195346498s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312324028 (375h26m4.195737661s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312707561 (375h26m4.196121192s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313124992 (375h26m4.19653862s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313485189 (375h26m4.196898821s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.893020764 (375h26m4.776434382s) [upload=300223]: Next returned nil segment
Jul 9 00:37:04.893957299 (375h26m4.777370932s) [upload=300223]: Waiting for error group managing segments...
Jul 9 00:37:05.037354849 (375h26m4.920768466s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.038444983 (375h26m4.921858597s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.039356590 (375h26m4.922770207s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.047559125 (375h26m4.930972742s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.053248590 (375h26m4.93666221s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.054452827 (375h26m4.937866447s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.889326616 (375h26m5.772740232s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:29.628225928 (375h26m29.511639541s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:30.256578837 (375h26m30.139992453s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:32.331498448 (375h26m32.214912065s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:39.883025841 (375h26m39.766439475s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:51.499841545 (375h26m51.383255165s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:55.982236621 (375h26m55.86565024s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607812826 (375h27m3.49122645s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607853389 (375h27m3.491267022s) [seg_pos={0 0}, upload=300223]: Segment reached optimal threshold of 14 pieces.
Jul 9 00:38:03.608006922 (375h27m3.491420552s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608045600 (375h27m3.491459233s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17LrYeCkrPWdiL9gGZ568PvfFct6cLVRQE4Qa92ykLgqXSfBL9): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608087309 (375h27m3.491500937s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RY6XGL5MJYc2idL6Np7DS3U9pCQGDhuEEqguY8AgjasjE4hiZ): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608191697 (375h27m3.491605324s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T6FqqMunZ46QoxhHVWsn8NinZGWv1Yo7rNHSiK6s9hhuQtS6Y): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608213412 (375h27m3.49162703s) [seg_pos={0 0}, upload=300223]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 00:38:03.608310462 (375h27m3.49172408s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:38:03.608316810 (375h27m3.491730438s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:03.761052568 (375h27m3.6444662s) [upload=300223]: Done uploading segments...
Jul 9 00:38:03.761076757 (375h27m3.644490378s) [upload=300223]: Upload finished. err: <nil>
Duration: 19.66927059s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........................X..X.............................] duration:
2.514536108s
FAIL (2025-07-09 01:00:48 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 10)
======LOGS======
entCommit
Jul 9 00:38:54.141867066 (375h27m54.025280684s) [upload=300248]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:54.286588243 (375h27m54.170001881s) [upload=300248]: Done uploading segments...
Jul 9 00:38:54.286633475 (375h27m54.170047105s) [upload=300248]: Upload finished. err: <nil>
Jul 9 00:39:49.853458099 (375h28m49.736871729s) [upload=300262]: Starting upload
Jul 9 00:39:49.853564556 (375h28m49.736978174s) [upload=300262]: Done starting upload
Jul 9 00:39:49.853754759 (375h28m49.737168394s) [upload=300262]: Uploading segments...
Jul 9 00:39:49.854072231 (375h28m49.737485857s) [upload=300262]: Got next segment. Inline: false
Jul 9 00:39:49.854139888 (375h28m49.737553515s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:39:49.854145670 (375h28m49.737559297s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:39:49.964717413 (375h28m49.848131039s) [seg_pos={0 0}, upload=300262]: Begin upload segment...
Jul 9 00:39:49.964772224 (375h28m49.848185839s) [seg_pos={0 0}, upload=300262]: Done begin upload segment.
Jul 9 00:39:49.964909767 (375h28m49.848323393s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965422318 (375h28m49.848835947s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965585955 (375h28m49.848999581s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966217091 (375h28m49.849630727s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966292916 (375h28m49.849706542s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967093655 (375h28m49.85050729s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967169721 (375h28m49.850583348s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967940005 (375h28m49.851353634s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.968588971 (375h28m49.8520026s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969080450 (375h28m49.852494085s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969563775 (375h28m49.852977409s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.970736354 (375h28m49.854149991s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.971229734 (375h28m49.85464336s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.972867679 (375h28m49.856281317s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.978267895 (375h28m49.861681529s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.986076032 (375h28m49.869489662s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:50.595435555 (375h28m50.478849173s) [upload=300262]: Next returned nil segment
Jul 9 00:39:50.595455642 (375h28m50.478869272s) [upload=300262]: Waiting for error group managing segments...
Jul 9 00:39:50.730628340 (375h28m50.614041956s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.731179161 (375h28m50.614592788s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.732951876 (375h28m50.616365493s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.738403976 (375h28m50.621817592s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739011956 (375h28m50.622425573s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739658988 (375h28m50.623072606s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.766298764 (375h28m50.649712392s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.754942777 (375h28m51.638356391s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.990915358 (375h28m51.87432899s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:13.196278807 (375h29m13.079692437s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:19.575930850 (375h29m19.459344476s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576005813 (375h29m19.459419439s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576051734 (375h29m19.459465361s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576066765 (375h29m19.459480398s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576255242 (375h29m19.459668869s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection
Jul 9 00:40:19.576293041 (375h29m19.459706667s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection
Jul 9 00:40:19.576316511 (375h29m19.45973014s) [seg_pos={0 0}, upload=300262]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil>
Jul 9 00:40:19.685159343 (375h29m19.568572971s) [upload=300262]: Done uploading segments...
Jul 9 00:40:19.685187481 (375h29m19.568601108s) [upload=300262]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10)
Duration: 2.514578301s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
15 out of 60 is failed (threshold=3) [...........X...........XX.X..X..X..XX.X...........XXXXX...X.] duration:
49.702430543s
FAIL (2025-07-09 01:00:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 49.702442519s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
26.606691495s
FAIL (2025-07-09 01:00:09 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
3980 (375h26m3.9562776s) [upload=300223]: Starting upload
Jul 9 00:37:04.072968784 (375h26m3.956382412s) [upload=300223]: Done starting upload
Jul 9 00:37:04.073087735 (375h26m3.956501365s) [upload=300223]: Uploading segments...
Jul 9 00:37:04.073377778 (375h26m3.956791396s) [upload=300223]: Got next segment. Inline: false
Jul 9 00:37:04.073430757 (375h26m3.956844388s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:37:04.073437151 (375h26m3.956850769s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:37:04.304692457 (375h26m4.18810608s) [seg_pos={0 0}, upload=300223]: Begin upload segment...
Jul 9 00:37:04.304761975 (375h26m4.188175587s) [seg_pos={0 0}, upload=300223]: Done begin upload segment.
Jul 9 00:37:04.304833908 (375h26m4.18824754s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305365539 (375h26m4.18877917s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305738725 (375h26m4.189152359s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305907885 (375h26m4.189321515s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306301693 (375h26m4.189715324s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306679055 (375h26m4.190092689s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307048455 (375h26m4.190462086s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307412956 (375h26m4.19082659s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.308062976 (375h26m4.191476609s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.309126547 (375h26m4.192540184s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.310676166 (375h26m4.194089804s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311146138 (375h26m4.19455977s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311543981 (375h26m4.194957595s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311932868 (375h26m4.195346498s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312324028 (375h26m4.195737661s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312707561 (375h26m4.196121192s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313124992 (375h26m4.19653862s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313485189 (375h26m4.196898821s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.893020764 (375h26m4.776434382s) [upload=300223]: Next returned nil segment
Jul 9 00:37:04.893957299 (375h26m4.777370932s) [upload=300223]: Waiting for error group managing segments...
Jul 9 00:37:05.037354849 (375h26m4.920768466s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.038444983 (375h26m4.921858597s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.039356590 (375h26m4.922770207s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.047559125 (375h26m4.930972742s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.053248590 (375h26m4.93666221s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.054452827 (375h26m4.937866447s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.889326616 (375h26m5.772740232s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:29.628225928 (375h26m29.511639541s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:30.256578837 (375h26m30.139992453s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:32.331498448 (375h26m32.214912065s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:39.883025841 (375h26m39.766439475s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:51.499841545 (375h26m51.383255165s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:55.982236621 (375h26m55.86565024s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607812826 (375h27m3.49122645s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607853389 (375h27m3.491267022s) [seg_pos={0 0}, upload=300223]: Segment reached optimal threshold of 14 pieces.
Jul 9 00:38:03.608006922 (375h27m3.491420552s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608045600 (375h27m3.491459233s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17LrYeCkrPWdiL9gGZ568PvfFct6cLVRQE4Qa92ykLgqXSfBL9): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608087309 (375h27m3.491500937s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RY6XGL5MJYc2idL6Np7DS3U9pCQGDhuEEqguY8AgjasjE4hiZ): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608191697 (375h27m3.491605324s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T6FqqMunZ46QoxhHVWsn8NinZGWv1Yo7rNHSiK6s9hhuQtS6Y): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608213412 (375h27m3.49162703s) [seg_pos={0 0}, upload=300223]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 00:38:03.608310462 (375h27m3.49172408s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:38:03.608316810 (375h27m3.491730438s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:03.761052568 (375h27m3.6444662s) [upload=300223]: Done uploading segments...
Jul 9 00:38:03.761076757 (375h27m3.644490378s) [upload=300223]: Upload finished. err: <nil>
Duration: 26.606722649s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [............................X..X............................] duration:
2.559211991s
FAIL (2025-07-09 01:00:08 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 10)
======LOGS======
entCommit
Jul 9 00:38:54.141867066 (375h27m54.025280684s) [upload=300248]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:54.286588243 (375h27m54.170001881s) [upload=300248]: Done uploading segments...
Jul 9 00:38:54.286633475 (375h27m54.170047105s) [upload=300248]: Upload finished. err: <nil>
Jul 9 00:39:49.853458099 (375h28m49.736871729s) [upload=300262]: Starting upload
Jul 9 00:39:49.853564556 (375h28m49.736978174s) [upload=300262]: Done starting upload
Jul 9 00:39:49.853754759 (375h28m49.737168394s) [upload=300262]: Uploading segments...
Jul 9 00:39:49.854072231 (375h28m49.737485857s) [upload=300262]: Got next segment. Inline: false
Jul 9 00:39:49.854139888 (375h28m49.737553515s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:39:49.854145670 (375h28m49.737559297s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:39:49.964717413 (375h28m49.848131039s) [seg_pos={0 0}, upload=300262]: Begin upload segment...
Jul 9 00:39:49.964772224 (375h28m49.848185839s) [seg_pos={0 0}, upload=300262]: Done begin upload segment.
Jul 9 00:39:49.964909767 (375h28m49.848323393s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965422318 (375h28m49.848835947s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965585955 (375h28m49.848999581s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966217091 (375h28m49.849630727s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966292916 (375h28m49.849706542s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967093655 (375h28m49.85050729s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967169721 (375h28m49.850583348s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967940005 (375h28m49.851353634s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.968588971 (375h28m49.8520026s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969080450 (375h28m49.852494085s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969563775 (375h28m49.852977409s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.970736354 (375h28m49.854149991s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.971229734 (375h28m49.85464336s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.972867679 (375h28m49.856281317s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.978267895 (375h28m49.861681529s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.986076032 (375h28m49.869489662s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:50.595435555 (375h28m50.478849173s) [upload=300262]: Next returned nil segment
Jul 9 00:39:50.595455642 (375h28m50.478869272s) [upload=300262]: Waiting for error group managing segments...
Jul 9 00:39:50.730628340 (375h28m50.614041956s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.731179161 (375h28m50.614592788s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.732951876 (375h28m50.616365493s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.738403976 (375h28m50.621817592s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739011956 (375h28m50.622425573s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739658988 (375h28m50.623072606s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.766298764 (375h28m50.649712392s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.754942777 (375h28m51.638356391s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.990915358 (375h28m51.87432899s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:13.196278807 (375h29m13.079692437s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:19.575930850 (375h29m19.459344476s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576005813 (375h29m19.459419439s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576051734 (375h29m19.459465361s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576066765 (375h29m19.459480398s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576255242 (375h29m19.459668869s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection
Jul 9 00:40:19.576293041 (375h29m19.459706667s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection
Jul 9 00:40:19.576316511 (375h29m19.45973014s) [seg_pos={0 0}, upload=300262]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil>
Jul 9 00:40:19.685159343 (375h29m19.568572971s) [upload=300262]: Done uploading segments...
Jul 9 00:40:19.685187481 (375h29m19.568601108s) [upload=300262]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10)
Duration: 2.559252276s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
15 out of 60 is failed (threshold=3) [............X...........XX.X..X..X..XX.X...........XXXXX...X] duration:
1m0.001295883s
FAIL (2025-07-09 00:59:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001309421s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............................X..X...........................] duration:
2.48954498s
FAIL (2025-07-09 00:59:33 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 10)
======LOGS======
entCommit
Jul 9 00:38:54.141867066 (375h27m54.025280684s) [upload=300248]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:54.286588243 (375h27m54.170001881s) [upload=300248]: Done uploading segments...
Jul 9 00:38:54.286633475 (375h27m54.170047105s) [upload=300248]: Upload finished. err: <nil>
Jul 9 00:39:49.853458099 (375h28m49.736871729s) [upload=300262]: Starting upload
Jul 9 00:39:49.853564556 (375h28m49.736978174s) [upload=300262]: Done starting upload
Jul 9 00:39:49.853754759 (375h28m49.737168394s) [upload=300262]: Uploading segments...
Jul 9 00:39:49.854072231 (375h28m49.737485857s) [upload=300262]: Got next segment. Inline: false
Jul 9 00:39:49.854139888 (375h28m49.737553515s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:39:49.854145670 (375h28m49.737559297s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:39:49.964717413 (375h28m49.848131039s) [seg_pos={0 0}, upload=300262]: Begin upload segment...
Jul 9 00:39:49.964772224 (375h28m49.848185839s) [seg_pos={0 0}, upload=300262]: Done begin upload segment.
Jul 9 00:39:49.964909767 (375h28m49.848323393s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965422318 (375h28m49.848835947s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965585955 (375h28m49.848999581s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966217091 (375h28m49.849630727s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966292916 (375h28m49.849706542s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967093655 (375h28m49.85050729s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967169721 (375h28m49.850583348s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967940005 (375h28m49.851353634s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.968588971 (375h28m49.8520026s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969080450 (375h28m49.852494085s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969563775 (375h28m49.852977409s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.970736354 (375h28m49.854149991s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.971229734 (375h28m49.85464336s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.972867679 (375h28m49.856281317s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.978267895 (375h28m49.861681529s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.986076032 (375h28m49.869489662s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:50.595435555 (375h28m50.478849173s) [upload=300262]: Next returned nil segment
Jul 9 00:39:50.595455642 (375h28m50.478869272s) [upload=300262]: Waiting for error group managing segments...
Jul 9 00:39:50.730628340 (375h28m50.614041956s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.731179161 (375h28m50.614592788s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.732951876 (375h28m50.616365493s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.738403976 (375h28m50.621817592s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739011956 (375h28m50.622425573s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739658988 (375h28m50.623072606s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.766298764 (375h28m50.649712392s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.754942777 (375h28m51.638356391s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.990915358 (375h28m51.87432899s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:13.196278807 (375h29m13.079692437s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:19.575930850 (375h29m19.459344476s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576005813 (375h29m19.459419439s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576051734 (375h29m19.459465361s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576066765 (375h29m19.459480398s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576255242 (375h29m19.459668869s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection
Jul 9 00:40:19.576293041 (375h29m19.459706667s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection
Jul 9 00:40:19.576316511 (375h29m19.45973014s) [seg_pos={0 0}, upload=300262]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil>
Jul 9 00:40:19.685159343 (375h29m19.568572971s) [upload=300262]: Done uploading segments...
Jul 9 00:40:19.685187481 (375h29m19.568601108s) [upload=300262]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10)
Duration: 2.489578751s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
10 out of 60 is failed (threshold=3) [..X..............X....X..X......X.............XX.X.X......X.] duration:
19.831666726s
FAIL (2025-07-09 00:59:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 19.831694895s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
20.237610864s
FAIL (2025-07-09 00:58:51 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
3980 (375h26m3.9562776s) [upload=300223]: Starting upload
Jul 9 00:37:04.072968784 (375h26m3.956382412s) [upload=300223]: Done starting upload
Jul 9 00:37:04.073087735 (375h26m3.956501365s) [upload=300223]: Uploading segments...
Jul 9 00:37:04.073377778 (375h26m3.956791396s) [upload=300223]: Got next segment. Inline: false
Jul 9 00:37:04.073430757 (375h26m3.956844388s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:37:04.073437151 (375h26m3.956850769s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:37:04.304692457 (375h26m4.18810608s) [seg_pos={0 0}, upload=300223]: Begin upload segment...
Jul 9 00:37:04.304761975 (375h26m4.188175587s) [seg_pos={0 0}, upload=300223]: Done begin upload segment.
Jul 9 00:37:04.304833908 (375h26m4.18824754s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305365539 (375h26m4.18877917s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305738725 (375h26m4.189152359s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305907885 (375h26m4.189321515s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306301693 (375h26m4.189715324s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306679055 (375h26m4.190092689s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307048455 (375h26m4.190462086s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307412956 (375h26m4.19082659s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.308062976 (375h26m4.191476609s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.309126547 (375h26m4.192540184s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.310676166 (375h26m4.194089804s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311146138 (375h26m4.19455977s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311543981 (375h26m4.194957595s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311932868 (375h26m4.195346498s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312324028 (375h26m4.195737661s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312707561 (375h26m4.196121192s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313124992 (375h26m4.19653862s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313485189 (375h26m4.196898821s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.893020764 (375h26m4.776434382s) [upload=300223]: Next returned nil segment
Jul 9 00:37:04.893957299 (375h26m4.777370932s) [upload=300223]: Waiting for error group managing segments...
Jul 9 00:37:05.037354849 (375h26m4.920768466s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.038444983 (375h26m4.921858597s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.039356590 (375h26m4.922770207s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.047559125 (375h26m4.930972742s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.053248590 (375h26m4.93666221s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.054452827 (375h26m4.937866447s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.889326616 (375h26m5.772740232s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:29.628225928 (375h26m29.511639541s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:30.256578837 (375h26m30.139992453s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:32.331498448 (375h26m32.214912065s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:39.883025841 (375h26m39.766439475s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:51.499841545 (375h26m51.383255165s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:55.982236621 (375h26m55.86565024s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607812826 (375h27m3.49122645s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607853389 (375h27m3.491267022s) [seg_pos={0 0}, upload=300223]: Segment reached optimal threshold of 14 pieces.
Jul 9 00:38:03.608006922 (375h27m3.491420552s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608045600 (375h27m3.491459233s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17LrYeCkrPWdiL9gGZ568PvfFct6cLVRQE4Qa92ykLgqXSfBL9): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608087309 (375h27m3.491500937s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RY6XGL5MJYc2idL6Np7DS3U9pCQGDhuEEqguY8AgjasjE4hiZ): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608191697 (375h27m3.491605324s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T6FqqMunZ46QoxhHVWsn8NinZGWv1Yo7rNHSiK6s9hhuQtS6Y): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608213412 (375h27m3.49162703s) [seg_pos={0 0}, upload=300223]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 00:38:03.608310462 (375h27m3.49172408s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:38:03.608316810 (375h27m3.491730438s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:03.761052568 (375h27m3.6444662s) [upload=300223]: Done uploading segments...
Jul 9 00:38:03.761076757 (375h27m3.644490378s) [upload=300223]: Upload finished. err: <nil>
Duration: 20.237653192s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
14 out of 60 is failed (threshold=3) [.............X...........XX.X..X..X..XX.X...........XXXXX...] duration:
55.540768259s
FAIL (2025-07-09 00:58:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 55.54078804s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
10 out of 60 is failed (threshold=3) [...X..............X....X..X......X.............XX.X.X......X] duration:
30.001667872s
FAIL (2025-07-09 00:58:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001695947s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............................X..X..........................] duration:
2.327722021s
FAIL (2025-07-09 00:58:39 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 10)
======LOGS======
entCommit
Jul 9 00:38:54.141867066 (375h27m54.025280684s) [upload=300248]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:54.286588243 (375h27m54.170001881s) [upload=300248]: Done uploading segments...
Jul 9 00:38:54.286633475 (375h27m54.170047105s) [upload=300248]: Upload finished. err: <nil>
Jul 9 00:39:49.853458099 (375h28m49.736871729s) [upload=300262]: Starting upload
Jul 9 00:39:49.853564556 (375h28m49.736978174s) [upload=300262]: Done starting upload
Jul 9 00:39:49.853754759 (375h28m49.737168394s) [upload=300262]: Uploading segments...
Jul 9 00:39:49.854072231 (375h28m49.737485857s) [upload=300262]: Got next segment. Inline: false
Jul 9 00:39:49.854139888 (375h28m49.737553515s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:39:49.854145670 (375h28m49.737559297s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:39:49.964717413 (375h28m49.848131039s) [seg_pos={0 0}, upload=300262]: Begin upload segment...
Jul 9 00:39:49.964772224 (375h28m49.848185839s) [seg_pos={0 0}, upload=300262]: Done begin upload segment.
Jul 9 00:39:49.964909767 (375h28m49.848323393s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965422318 (375h28m49.848835947s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965585955 (375h28m49.848999581s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966217091 (375h28m49.849630727s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966292916 (375h28m49.849706542s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967093655 (375h28m49.85050729s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967169721 (375h28m49.850583348s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967940005 (375h28m49.851353634s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.968588971 (375h28m49.8520026s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969080450 (375h28m49.852494085s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969563775 (375h28m49.852977409s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.970736354 (375h28m49.854149991s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.971229734 (375h28m49.85464336s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.972867679 (375h28m49.856281317s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.978267895 (375h28m49.861681529s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.986076032 (375h28m49.869489662s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:50.595435555 (375h28m50.478849173s) [upload=300262]: Next returned nil segment
Jul 9 00:39:50.595455642 (375h28m50.478869272s) [upload=300262]: Waiting for error group managing segments...
Jul 9 00:39:50.730628340 (375h28m50.614041956s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.731179161 (375h28m50.614592788s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.732951876 (375h28m50.616365493s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.738403976 (375h28m50.621817592s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739011956 (375h28m50.622425573s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739658988 (375h28m50.623072606s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.766298764 (375h28m50.649712392s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.754942777 (375h28m51.638356391s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.990915358 (375h28m51.87432899s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:13.196278807 (375h29m13.079692437s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:19.575930850 (375h29m19.459344476s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576005813 (375h29m19.459419439s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576051734 (375h29m19.459465361s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576066765 (375h29m19.459480398s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576255242 (375h29m19.459668869s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection
Jul 9 00:40:19.576293041 (375h29m19.459706667s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection
Jul 9 00:40:19.576316511 (375h29m19.45973014s) [seg_pos={0 0}, upload=300262]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil>
Jul 9 00:40:19.685159343 (375h29m19.568572971s) [upload=300262]: Done uploading segments...
Jul 9 00:40:19.685187481 (375h29m19.568601108s) [upload=300262]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10)
Duration: 2.32775473s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
23.393532414s
FAIL (2025-07-09 00:58:16 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
3980 (375h26m3.9562776s) [upload=300223]: Starting upload
Jul 9 00:37:04.072968784 (375h26m3.956382412s) [upload=300223]: Done starting upload
Jul 9 00:37:04.073087735 (375h26m3.956501365s) [upload=300223]: Uploading segments...
Jul 9 00:37:04.073377778 (375h26m3.956791396s) [upload=300223]: Got next segment. Inline: false
Jul 9 00:37:04.073430757 (375h26m3.956844388s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:37:04.073437151 (375h26m3.956850769s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:37:04.304692457 (375h26m4.18810608s) [seg_pos={0 0}, upload=300223]: Begin upload segment...
Jul 9 00:37:04.304761975 (375h26m4.188175587s) [seg_pos={0 0}, upload=300223]: Done begin upload segment.
Jul 9 00:37:04.304833908 (375h26m4.18824754s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305365539 (375h26m4.18877917s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305738725 (375h26m4.189152359s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305907885 (375h26m4.189321515s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306301693 (375h26m4.189715324s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306679055 (375h26m4.190092689s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307048455 (375h26m4.190462086s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307412956 (375h26m4.19082659s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.308062976 (375h26m4.191476609s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.309126547 (375h26m4.192540184s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.310676166 (375h26m4.194089804s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311146138 (375h26m4.19455977s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311543981 (375h26m4.194957595s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311932868 (375h26m4.195346498s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312324028 (375h26m4.195737661s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312707561 (375h26m4.196121192s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313124992 (375h26m4.19653862s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313485189 (375h26m4.196898821s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.893020764 (375h26m4.776434382s) [upload=300223]: Next returned nil segment
Jul 9 00:37:04.893957299 (375h26m4.777370932s) [upload=300223]: Waiting for error group managing segments...
Jul 9 00:37:05.037354849 (375h26m4.920768466s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.038444983 (375h26m4.921858597s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.039356590 (375h26m4.922770207s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.047559125 (375h26m4.930972742s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.053248590 (375h26m4.93666221s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.054452827 (375h26m4.937866447s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.889326616 (375h26m5.772740232s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:29.628225928 (375h26m29.511639541s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:30.256578837 (375h26m30.139992453s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:32.331498448 (375h26m32.214912065s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:39.883025841 (375h26m39.766439475s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:51.499841545 (375h26m51.383255165s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:55.982236621 (375h26m55.86565024s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607812826 (375h27m3.49122645s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607853389 (375h27m3.491267022s) [seg_pos={0 0}, upload=300223]: Segment reached optimal threshold of 14 pieces.
Jul 9 00:38:03.608006922 (375h27m3.491420552s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608045600 (375h27m3.491459233s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17LrYeCkrPWdiL9gGZ568PvfFct6cLVRQE4Qa92ykLgqXSfBL9): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608087309 (375h27m3.491500937s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RY6XGL5MJYc2idL6Np7DS3U9pCQGDhuEEqguY8AgjasjE4hiZ): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608191697 (375h27m3.491605324s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T6FqqMunZ46QoxhHVWsn8NinZGWv1Yo7rNHSiK6s9hhuQtS6Y): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608213412 (375h27m3.49162703s) [seg_pos={0 0}, upload=300223]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 00:38:03.608310462 (375h27m3.49172408s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:38:03.608316810 (375h27m3.491730438s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:03.761052568 (375h27m3.6444662s) [upload=300223]: Done uploading segments...
Jul 9 00:38:03.761076757 (375h27m3.644490378s) [upload=300223]: Upload finished. err: <nil>
Duration: 23.393563391s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............................X..X.........................] duration:
2.40281812s
FAIL (2025-07-09 00:58:11 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 10)
======LOGS======
entCommit
Jul 9 00:38:54.141867066 (375h27m54.025280684s) [upload=300248]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:54.286588243 (375h27m54.170001881s) [upload=300248]: Done uploading segments...
Jul 9 00:38:54.286633475 (375h27m54.170047105s) [upload=300248]: Upload finished. err: <nil>
Jul 9 00:39:49.853458099 (375h28m49.736871729s) [upload=300262]: Starting upload
Jul 9 00:39:49.853564556 (375h28m49.736978174s) [upload=300262]: Done starting upload
Jul 9 00:39:49.853754759 (375h28m49.737168394s) [upload=300262]: Uploading segments...
Jul 9 00:39:49.854072231 (375h28m49.737485857s) [upload=300262]: Got next segment. Inline: false
Jul 9 00:39:49.854139888 (375h28m49.737553515s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:39:49.854145670 (375h28m49.737559297s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:39:49.964717413 (375h28m49.848131039s) [seg_pos={0 0}, upload=300262]: Begin upload segment...
Jul 9 00:39:49.964772224 (375h28m49.848185839s) [seg_pos={0 0}, upload=300262]: Done begin upload segment.
Jul 9 00:39:49.964909767 (375h28m49.848323393s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965422318 (375h28m49.848835947s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965585955 (375h28m49.848999581s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966217091 (375h28m49.849630727s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966292916 (375h28m49.849706542s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967093655 (375h28m49.85050729s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967169721 (375h28m49.850583348s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967940005 (375h28m49.851353634s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.968588971 (375h28m49.8520026s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969080450 (375h28m49.852494085s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969563775 (375h28m49.852977409s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.970736354 (375h28m49.854149991s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.971229734 (375h28m49.85464336s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.972867679 (375h28m49.856281317s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.978267895 (375h28m49.861681529s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.986076032 (375h28m49.869489662s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:50.595435555 (375h28m50.478849173s) [upload=300262]: Next returned nil segment
Jul 9 00:39:50.595455642 (375h28m50.478869272s) [upload=300262]: Waiting for error group managing segments...
Jul 9 00:39:50.730628340 (375h28m50.614041956s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.731179161 (375h28m50.614592788s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.732951876 (375h28m50.616365493s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.738403976 (375h28m50.621817592s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739011956 (375h28m50.622425573s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739658988 (375h28m50.623072606s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.766298764 (375h28m50.649712392s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.754942777 (375h28m51.638356391s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.990915358 (375h28m51.87432899s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:13.196278807 (375h29m13.079692437s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:19.575930850 (375h29m19.459344476s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576005813 (375h29m19.459419439s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576051734 (375h29m19.459465361s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576066765 (375h29m19.459480398s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576255242 (375h29m19.459668869s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection
Jul 9 00:40:19.576293041 (375h29m19.459706667s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection
Jul 9 00:40:19.576316511 (375h29m19.45973014s) [seg_pos={0 0}, upload=300262]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil>
Jul 9 00:40:19.685159343 (375h29m19.568572971s) [upload=300262]: Done uploading segments...
Jul 9 00:40:19.685187481 (375h29m19.568601108s) [upload=300262]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10)
Duration: 2.402862958s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
14 out of 60 is failed (threshold=3) [..............X...........XX.X..X..X..XX.X...........XXXXX..] duration:
37.2201417s
FAIL (2025-07-09 00:57:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 37.220158045s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [................................X..X........................] duration:
5.869881434s
FAIL (2025-07-09 00:57:24 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 10)
======LOGS======
entCommit
Jul 9 00:38:54.141867066 (375h27m54.025280684s) [upload=300248]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:54.286588243 (375h27m54.170001881s) [upload=300248]: Done uploading segments...
Jul 9 00:38:54.286633475 (375h27m54.170047105s) [upload=300248]: Upload finished. err: <nil>
Jul 9 00:39:49.853458099 (375h28m49.736871729s) [upload=300262]: Starting upload
Jul 9 00:39:49.853564556 (375h28m49.736978174s) [upload=300262]: Done starting upload
Jul 9 00:39:49.853754759 (375h28m49.737168394s) [upload=300262]: Uploading segments...
Jul 9 00:39:49.854072231 (375h28m49.737485857s) [upload=300262]: Got next segment. Inline: false
Jul 9 00:39:49.854139888 (375h28m49.737553515s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:39:49.854145670 (375h28m49.737559297s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:39:49.964717413 (375h28m49.848131039s) [seg_pos={0 0}, upload=300262]: Begin upload segment...
Jul 9 00:39:49.964772224 (375h28m49.848185839s) [seg_pos={0 0}, upload=300262]: Done begin upload segment.
Jul 9 00:39:49.964909767 (375h28m49.848323393s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965422318 (375h28m49.848835947s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965585955 (375h28m49.848999581s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966217091 (375h28m49.849630727s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966292916 (375h28m49.849706542s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967093655 (375h28m49.85050729s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967169721 (375h28m49.850583348s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967940005 (375h28m49.851353634s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.968588971 (375h28m49.8520026s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969080450 (375h28m49.852494085s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969563775 (375h28m49.852977409s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.970736354 (375h28m49.854149991s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.971229734 (375h28m49.85464336s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.972867679 (375h28m49.856281317s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.978267895 (375h28m49.861681529s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.986076032 (375h28m49.869489662s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:50.595435555 (375h28m50.478849173s) [upload=300262]: Next returned nil segment
Jul 9 00:39:50.595455642 (375h28m50.478869272s) [upload=300262]: Waiting for error group managing segments...
Jul 9 00:39:50.730628340 (375h28m50.614041956s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.731179161 (375h28m50.614592788s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.732951876 (375h28m50.616365493s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.738403976 (375h28m50.621817592s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739011956 (375h28m50.622425573s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739658988 (375h28m50.623072606s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.766298764 (375h28m50.649712392s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.754942777 (375h28m51.638356391s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.990915358 (375h28m51.87432899s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:13.196278807 (375h29m13.079692437s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:19.575930850 (375h29m19.459344476s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576005813 (375h29m19.459419439s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576051734 (375h29m19.459465361s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576066765 (375h29m19.459480398s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576255242 (375h29m19.459668869s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection
Jul 9 00:40:19.576293041 (375h29m19.459706667s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection
Jul 9 00:40:19.576316511 (375h29m19.45973014s) [seg_pos={0 0}, upload=300262]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil>
Jul 9 00:40:19.685159343 (375h29m19.568572971s) [upload=300262]: Done uploading segments...
Jul 9 00:40:19.685187481 (375h29m19.568601108s) [upload=300262]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10)
Duration: 5.869953368s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
10 out of 60 is failed (threshold=3) [X...X..............X....X..X......X.............XX.X.X......] duration:
3.228666472s
FAIL (2025-07-09 00:57:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.228699875s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
14 out of 60 is failed (threshold=3) [...............X...........XX.X..X..X..XX.X...........XXXXX.] duration:
33.388316388s
FAIL (2025-07-09 00:57:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 33.388343461s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
12.692136861s
FAIL (2025-07-09 00:57:05 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
3980 (375h26m3.9562776s) [upload=300223]: Starting upload
Jul 9 00:37:04.072968784 (375h26m3.956382412s) [upload=300223]: Done starting upload
Jul 9 00:37:04.073087735 (375h26m3.956501365s) [upload=300223]: Uploading segments...
Jul 9 00:37:04.073377778 (375h26m3.956791396s) [upload=300223]: Got next segment. Inline: false
Jul 9 00:37:04.073430757 (375h26m3.956844388s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:37:04.073437151 (375h26m3.956850769s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:37:04.304692457 (375h26m4.18810608s) [seg_pos={0 0}, upload=300223]: Begin upload segment...
Jul 9 00:37:04.304761975 (375h26m4.188175587s) [seg_pos={0 0}, upload=300223]: Done begin upload segment.
Jul 9 00:37:04.304833908 (375h26m4.18824754s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305365539 (375h26m4.18877917s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305738725 (375h26m4.189152359s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305907885 (375h26m4.189321515s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306301693 (375h26m4.189715324s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306679055 (375h26m4.190092689s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307048455 (375h26m4.190462086s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307412956 (375h26m4.19082659s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.308062976 (375h26m4.191476609s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.309126547 (375h26m4.192540184s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.310676166 (375h26m4.194089804s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311146138 (375h26m4.19455977s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311543981 (375h26m4.194957595s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311932868 (375h26m4.195346498s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312324028 (375h26m4.195737661s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312707561 (375h26m4.196121192s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313124992 (375h26m4.19653862s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313485189 (375h26m4.196898821s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.893020764 (375h26m4.776434382s) [upload=300223]: Next returned nil segment
Jul 9 00:37:04.893957299 (375h26m4.777370932s) [upload=300223]: Waiting for error group managing segments...
Jul 9 00:37:05.037354849 (375h26m4.920768466s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.038444983 (375h26m4.921858597s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.039356590 (375h26m4.922770207s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.047559125 (375h26m4.930972742s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.053248590 (375h26m4.93666221s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.054452827 (375h26m4.937866447s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.889326616 (375h26m5.772740232s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:29.628225928 (375h26m29.511639541s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:30.256578837 (375h26m30.139992453s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:32.331498448 (375h26m32.214912065s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:39.883025841 (375h26m39.766439475s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:51.499841545 (375h26m51.383255165s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:55.982236621 (375h26m55.86565024s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607812826 (375h27m3.49122645s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607853389 (375h27m3.491267022s) [seg_pos={0 0}, upload=300223]: Segment reached optimal threshold of 14 pieces.
Jul 9 00:38:03.608006922 (375h27m3.491420552s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608045600 (375h27m3.491459233s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17LrYeCkrPWdiL9gGZ568PvfFct6cLVRQE4Qa92ykLgqXSfBL9): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608087309 (375h27m3.491500937s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RY6XGL5MJYc2idL6Np7DS3U9pCQGDhuEEqguY8AgjasjE4hiZ): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608191697 (375h27m3.491605324s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T6FqqMunZ46QoxhHVWsn8NinZGWv1Yo7rNHSiK6s9hhuQtS6Y): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608213412 (375h27m3.49162703s) [seg_pos={0 0}, upload=300223]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 00:38:03.608310462 (375h27m3.49172408s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:38:03.608316810 (375h27m3.491730438s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:03.761052568 (375h27m3.6444662s) [upload=300223]: Done uploading segments...
Jul 9 00:38:03.761076757 (375h27m3.644490378s) [upload=300223]: Upload finished. err: <nil>
Duration: 12.692169619s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................................X..X.......................] duration:
2.481649102s
FAIL (2025-07-09 00:56:56 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 10)
======LOGS======
entCommit
Jul 9 00:38:54.141867066 (375h27m54.025280684s) [upload=300248]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:54.286588243 (375h27m54.170001881s) [upload=300248]: Done uploading segments...
Jul 9 00:38:54.286633475 (375h27m54.170047105s) [upload=300248]: Upload finished. err: <nil>
Jul 9 00:39:49.853458099 (375h28m49.736871729s) [upload=300262]: Starting upload
Jul 9 00:39:49.853564556 (375h28m49.736978174s) [upload=300262]: Done starting upload
Jul 9 00:39:49.853754759 (375h28m49.737168394s) [upload=300262]: Uploading segments...
Jul 9 00:39:49.854072231 (375h28m49.737485857s) [upload=300262]: Got next segment. Inline: false
Jul 9 00:39:49.854139888 (375h28m49.737553515s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:39:49.854145670 (375h28m49.737559297s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:39:49.964717413 (375h28m49.848131039s) [seg_pos={0 0}, upload=300262]: Begin upload segment...
Jul 9 00:39:49.964772224 (375h28m49.848185839s) [seg_pos={0 0}, upload=300262]: Done begin upload segment.
Jul 9 00:39:49.964909767 (375h28m49.848323393s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965422318 (375h28m49.848835947s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965585955 (375h28m49.848999581s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966217091 (375h28m49.849630727s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966292916 (375h28m49.849706542s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967093655 (375h28m49.85050729s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967169721 (375h28m49.850583348s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967940005 (375h28m49.851353634s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.968588971 (375h28m49.8520026s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969080450 (375h28m49.852494085s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969563775 (375h28m49.852977409s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.970736354 (375h28m49.854149991s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.971229734 (375h28m49.85464336s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.972867679 (375h28m49.856281317s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.978267895 (375h28m49.861681529s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.986076032 (375h28m49.869489662s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:50.595435555 (375h28m50.478849173s) [upload=300262]: Next returned nil segment
Jul 9 00:39:50.595455642 (375h28m50.478869272s) [upload=300262]: Waiting for error group managing segments...
Jul 9 00:39:50.730628340 (375h28m50.614041956s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.731179161 (375h28m50.614592788s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.732951876 (375h28m50.616365493s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.738403976 (375h28m50.621817592s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739011956 (375h28m50.622425573s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739658988 (375h28m50.623072606s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.766298764 (375h28m50.649712392s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.754942777 (375h28m51.638356391s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.990915358 (375h28m51.87432899s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:13.196278807 (375h29m13.079692437s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:19.575930850 (375h29m19.459344476s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576005813 (375h29m19.459419439s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576051734 (375h29m19.459465361s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576066765 (375h29m19.459480398s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576255242 (375h29m19.459668869s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection
Jul 9 00:40:19.576293041 (375h29m19.459706667s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection
Jul 9 00:40:19.576316511 (375h29m19.45973014s) [seg_pos={0 0}, upload=300262]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil>
Jul 9 00:40:19.685159343 (375h29m19.568572971s) [upload=300262]: Done uploading segments...
Jul 9 00:40:19.685187481 (375h29m19.568601108s) [upload=300262]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10)
Duration: 2.481690053s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..................................X..X......................] duration:
27.326293727s
FAIL (2025-07-09 00:56:53 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 10)
======LOGS======
entCommit
Jul 9 00:38:54.141867066 (375h27m54.025280684s) [upload=300248]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:54.286588243 (375h27m54.170001881s) [upload=300248]: Done uploading segments...
Jul 9 00:38:54.286633475 (375h27m54.170047105s) [upload=300248]: Upload finished. err: <nil>
Jul 9 00:39:49.853458099 (375h28m49.736871729s) [upload=300262]: Starting upload
Jul 9 00:39:49.853564556 (375h28m49.736978174s) [upload=300262]: Done starting upload
Jul 9 00:39:49.853754759 (375h28m49.737168394s) [upload=300262]: Uploading segments...
Jul 9 00:39:49.854072231 (375h28m49.737485857s) [upload=300262]: Got next segment. Inline: false
Jul 9 00:39:49.854139888 (375h28m49.737553515s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:39:49.854145670 (375h28m49.737559297s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:39:49.964717413 (375h28m49.848131039s) [seg_pos={0 0}, upload=300262]: Begin upload segment...
Jul 9 00:39:49.964772224 (375h28m49.848185839s) [seg_pos={0 0}, upload=300262]: Done begin upload segment.
Jul 9 00:39:49.964909767 (375h28m49.848323393s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965422318 (375h28m49.848835947s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965585955 (375h28m49.848999581s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966217091 (375h28m49.849630727s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966292916 (375h28m49.849706542s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967093655 (375h28m49.85050729s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967169721 (375h28m49.850583348s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967940005 (375h28m49.851353634s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.968588971 (375h28m49.8520026s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969080450 (375h28m49.852494085s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969563775 (375h28m49.852977409s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.970736354 (375h28m49.854149991s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.971229734 (375h28m49.85464336s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.972867679 (375h28m49.856281317s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.978267895 (375h28m49.861681529s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.986076032 (375h28m49.869489662s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:50.595435555 (375h28m50.478849173s) [upload=300262]: Next returned nil segment
Jul 9 00:39:50.595455642 (375h28m50.478869272s) [upload=300262]: Waiting for error group managing segments...
Jul 9 00:39:50.730628340 (375h28m50.614041956s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.731179161 (375h28m50.614592788s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.732951876 (375h28m50.616365493s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.738403976 (375h28m50.621817592s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739011956 (375h28m50.622425573s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739658988 (375h28m50.623072606s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.766298764 (375h28m50.649712392s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.754942777 (375h28m51.638356391s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.990915358 (375h28m51.87432899s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:13.196278807 (375h29m13.079692437s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:19.575930850 (375h29m19.459344476s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576005813 (375h29m19.459419439s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576051734 (375h29m19.459465361s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576066765 (375h29m19.459480398s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576255242 (375h29m19.459668869s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection
Jul 9 00:40:19.576293041 (375h29m19.459706667s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection
Jul 9 00:40:19.576316511 (375h29m19.45973014s) [seg_pos={0 0}, upload=300262]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil>
Jul 9 00:40:19.685159343 (375h29m19.568572971s) [upload=300262]: Done uploading segments...
Jul 9 00:40:19.685187481 (375h29m19.568601108s) [upload=300262]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10)
Duration: 27.326323753s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
15.383525707s
FAIL (2025-07-09 00:56:50 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
3980 (375h26m3.9562776s) [upload=300223]: Starting upload
Jul 9 00:37:04.072968784 (375h26m3.956382412s) [upload=300223]: Done starting upload
Jul 9 00:37:04.073087735 (375h26m3.956501365s) [upload=300223]: Uploading segments...
Jul 9 00:37:04.073377778 (375h26m3.956791396s) [upload=300223]: Got next segment. Inline: false
Jul 9 00:37:04.073430757 (375h26m3.956844388s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:37:04.073437151 (375h26m3.956850769s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:37:04.304692457 (375h26m4.18810608s) [seg_pos={0 0}, upload=300223]: Begin upload segment...
Jul 9 00:37:04.304761975 (375h26m4.188175587s) [seg_pos={0 0}, upload=300223]: Done begin upload segment.
Jul 9 00:37:04.304833908 (375h26m4.18824754s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305365539 (375h26m4.18877917s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305738725 (375h26m4.189152359s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305907885 (375h26m4.189321515s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306301693 (375h26m4.189715324s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306679055 (375h26m4.190092689s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307048455 (375h26m4.190462086s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307412956 (375h26m4.19082659s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.308062976 (375h26m4.191476609s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.309126547 (375h26m4.192540184s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.310676166 (375h26m4.194089804s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311146138 (375h26m4.19455977s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311543981 (375h26m4.194957595s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311932868 (375h26m4.195346498s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312324028 (375h26m4.195737661s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312707561 (375h26m4.196121192s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313124992 (375h26m4.19653862s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313485189 (375h26m4.196898821s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.893020764 (375h26m4.776434382s) [upload=300223]: Next returned nil segment
Jul 9 00:37:04.893957299 (375h26m4.777370932s) [upload=300223]: Waiting for error group managing segments...
Jul 9 00:37:05.037354849 (375h26m4.920768466s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.038444983 (375h26m4.921858597s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.039356590 (375h26m4.922770207s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.047559125 (375h26m4.930972742s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.053248590 (375h26m4.93666221s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.054452827 (375h26m4.937866447s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.889326616 (375h26m5.772740232s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:29.628225928 (375h26m29.511639541s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:30.256578837 (375h26m30.139992453s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:32.331498448 (375h26m32.214912065s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:39.883025841 (375h26m39.766439475s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:51.499841545 (375h26m51.383255165s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:55.982236621 (375h26m55.86565024s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607812826 (375h27m3.49122645s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607853389 (375h27m3.491267022s) [seg_pos={0 0}, upload=300223]: Segment reached optimal threshold of 14 pieces.
Jul 9 00:38:03.608006922 (375h27m3.491420552s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608045600 (375h27m3.491459233s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17LrYeCkrPWdiL9gGZ568PvfFct6cLVRQE4Qa92ykLgqXSfBL9): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608087309 (375h27m3.491500937s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RY6XGL5MJYc2idL6Np7DS3U9pCQGDhuEEqguY8AgjasjE4hiZ): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608191697 (375h27m3.491605324s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T6FqqMunZ46QoxhHVWsn8NinZGWv1Yo7rNHSiK6s9hhuQtS6Y): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608213412 (375h27m3.49162703s) [seg_pos={0 0}, upload=300223]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 00:38:03.608310462 (375h27m3.49172408s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:38:03.608316810 (375h27m3.491730438s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:03.761052568 (375h27m3.6444662s) [upload=300223]: Done uploading segments...
Jul 9 00:38:03.761076757 (375h27m3.644490378s) [upload=300223]: Upload finished. err: <nil>
Duration: 15.383563329s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
14 out of 60 is failed (threshold=3) [................X...........XX.X..X..X..XX.X...........XXXXX] duration:
1m0.001444204s
FAIL (2025-07-09 00:56:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001463236s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
11 out of 60 is failed (threshold=3) [XX...X..............X....X..X......X.............XX.X.X.....] duration:
2.820352165s
FAIL (2025-07-09 00:56:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.820390791s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
20.584528682s
FAIL (2025-07-09 00:56:06 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
3980 (375h26m3.9562776s) [upload=300223]: Starting upload
Jul 9 00:37:04.072968784 (375h26m3.956382412s) [upload=300223]: Done starting upload
Jul 9 00:37:04.073087735 (375h26m3.956501365s) [upload=300223]: Uploading segments...
Jul 9 00:37:04.073377778 (375h26m3.956791396s) [upload=300223]: Got next segment. Inline: false
Jul 9 00:37:04.073430757 (375h26m3.956844388s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:37:04.073437151 (375h26m3.956850769s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:37:04.304692457 (375h26m4.18810608s) [seg_pos={0 0}, upload=300223]: Begin upload segment...
Jul 9 00:37:04.304761975 (375h26m4.188175587s) [seg_pos={0 0}, upload=300223]: Done begin upload segment.
Jul 9 00:37:04.304833908 (375h26m4.18824754s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305365539 (375h26m4.18877917s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305738725 (375h26m4.189152359s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305907885 (375h26m4.189321515s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306301693 (375h26m4.189715324s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306679055 (375h26m4.190092689s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307048455 (375h26m4.190462086s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307412956 (375h26m4.19082659s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.308062976 (375h26m4.191476609s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.309126547 (375h26m4.192540184s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.310676166 (375h26m4.194089804s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311146138 (375h26m4.19455977s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311543981 (375h26m4.194957595s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311932868 (375h26m4.195346498s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312324028 (375h26m4.195737661s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312707561 (375h26m4.196121192s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313124992 (375h26m4.19653862s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313485189 (375h26m4.196898821s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.893020764 (375h26m4.776434382s) [upload=300223]: Next returned nil segment
Jul 9 00:37:04.893957299 (375h26m4.777370932s) [upload=300223]: Waiting for error group managing segments...
Jul 9 00:37:05.037354849 (375h26m4.920768466s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.038444983 (375h26m4.921858597s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.039356590 (375h26m4.922770207s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.047559125 (375h26m4.930972742s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.053248590 (375h26m4.93666221s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.054452827 (375h26m4.937866447s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.889326616 (375h26m5.772740232s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:29.628225928 (375h26m29.511639541s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:30.256578837 (375h26m30.139992453s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:32.331498448 (375h26m32.214912065s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:39.883025841 (375h26m39.766439475s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:51.499841545 (375h26m51.383255165s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:55.982236621 (375h26m55.86565024s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607812826 (375h27m3.49122645s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607853389 (375h27m3.491267022s) [seg_pos={0 0}, upload=300223]: Segment reached optimal threshold of 14 pieces.
Jul 9 00:38:03.608006922 (375h27m3.491420552s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608045600 (375h27m3.491459233s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17LrYeCkrPWdiL9gGZ568PvfFct6cLVRQE4Qa92ykLgqXSfBL9): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608087309 (375h27m3.491500937s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RY6XGL5MJYc2idL6Np7DS3U9pCQGDhuEEqguY8AgjasjE4hiZ): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608191697 (375h27m3.491605324s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T6FqqMunZ46QoxhHVWsn8NinZGWv1Yo7rNHSiK6s9hhuQtS6Y): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608213412 (375h27m3.49162703s) [seg_pos={0 0}, upload=300223]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 00:38:03.608310462 (375h27m3.49172408s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:38:03.608316810 (375h27m3.491730438s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:03.761052568 (375h27m3.6444662s) [upload=300223]: Done uploading segments...
Jul 9 00:38:03.761076757 (375h27m3.644490378s) [upload=300223]: Upload finished. err: <nil>
Duration: 20.584555808s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
19.674874672s
FAIL (2025-07-09 00:55:45 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
3980 (375h26m3.9562776s) [upload=300223]: Starting upload
Jul 9 00:37:04.072968784 (375h26m3.956382412s) [upload=300223]: Done starting upload
Jul 9 00:37:04.073087735 (375h26m3.956501365s) [upload=300223]: Uploading segments...
Jul 9 00:37:04.073377778 (375h26m3.956791396s) [upload=300223]: Got next segment. Inline: false
Jul 9 00:37:04.073430757 (375h26m3.956844388s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:37:04.073437151 (375h26m3.956850769s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:37:04.304692457 (375h26m4.18810608s) [seg_pos={0 0}, upload=300223]: Begin upload segment...
Jul 9 00:37:04.304761975 (375h26m4.188175587s) [seg_pos={0 0}, upload=300223]: Done begin upload segment.
Jul 9 00:37:04.304833908 (375h26m4.18824754s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305365539 (375h26m4.18877917s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305738725 (375h26m4.189152359s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.305907885 (375h26m4.189321515s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306301693 (375h26m4.189715324s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.306679055 (375h26m4.190092689s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307048455 (375h26m4.190462086s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.307412956 (375h26m4.19082659s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.308062976 (375h26m4.191476609s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.309126547 (375h26m4.192540184s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.310676166 (375h26m4.194089804s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311146138 (375h26m4.19455977s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311543981 (375h26m4.194957595s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.311932868 (375h26m4.195346498s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312324028 (375h26m4.195737661s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.312707561 (375h26m4.196121192s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313124992 (375h26m4.19653862s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.313485189 (375h26m4.196898821s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Uploading piece...
Jul 9 00:37:04.893020764 (375h26m4.776434382s) [upload=300223]: Next returned nil segment
Jul 9 00:37:04.893957299 (375h26m4.777370932s) [upload=300223]: Waiting for error group managing segments...
Jul 9 00:37:05.037354849 (375h26m4.920768466s) [noise=true , address=77.90.31.10:4547 , piece_id=V3YTAVFJL7UPZYPJXQL3VWHNZHTQZBQ3P46BEULJGRT4MRI5OFTQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.038444983 (375h26m4.921858597s) [noise=true , address=77.90.31.16:4765 , piece_id=FMYOKA3GUMYAQJFLKULKN37AGXAQNMDUXVZTVA5M7UFM76XLQPXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.039356590 (375h26m4.922770207s) [noise=true , address=77.90.31.16:4763 , piece_id=PTKGN4O4BD5EZUFJ7DJYNEPYAB4RAXUCPZI7ZPKMGAUEXCA2Y7AA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.047559125 (375h26m4.930972742s) [noise=true , address=77.90.31.16:4736 , piece_id=MM6DFSLH64WYDRQUNNTABPUIQF7UI6RWIJRUELQGU33RDHEAXMRA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.053248590 (375h26m4.93666221s) [noise=true , address=77.90.31.17:4795 , piece_id=ABVYKQWDKVSSLA2GQWDSNXMIBFZRN6YLISO6VRANUA6HAEQAHYFQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.054452827 (375h26m4.937866447s) [noise=true , address=77.90.31.15:4727 , piece_id=IDT7ZUGMUHZQNFGIVHAKD2A7Z6I4726HZHDR2UZMQOBH7AOB27TQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:05.889326616 (375h26m5.772740232s) [noise=true , address=79.127.219.48:4076 , piece_id=KXP6PMGN5ZI2SIQKKXL23ENORCC274V6YRU57AVAPAOO5ZCZUBIQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:29.628225928 (375h26m29.511639541s) [noise=true , address=104.36.82.6:4898 , piece_id=YFHP6ZZY3FRU7Y3GWU5VDJ6YX3BGFZUR2WBDYVI24MZCE5CROJRQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:30.256578837 (375h26m30.139992453s) [noise=true , address=104.36.82.42:4817 , piece_id=CN3V3YEEJLS3U7ZCDOKQQRB5TZE2HKOJYF4NJAJJ2CAMXME7Q2ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:32.331498448 (375h26m32.214912065s) [noise=true , address=104.36.82.38:4708 , piece_id=MWDSQCSF2KOZFDEXUWK7BS6G2MYG4EIVOZDDZQPLDGSVXOLVJG7A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:39.883025841 (375h26m39.766439475s) [noise=true , address=104.36.82.35:4614 , piece_id=IS5NVS4QT3TNFR744BJXYYHSNBGNS425AAJLGHJYTYJQ4R44NB2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:51.499841545 (375h26m51.383255165s) [noise=true , address=104.36.82.60:4368 , piece_id=5C5C6KCHEC2ZSIBNAWRBEV442DKVID6ILQ7JA6DFFGB6BBV2EVXA, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:37:55.982236621 (375h26m55.86565024s) [noise=true , address=104.36.82.10:4091 , piece_id=2CZWVKEU2J2IEC3M4JTU5G2TZ2P2ERRLMID2CZR5X4L7G5WMZRYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607812826 (375h27m3.49122645s) [noise=true , address=104.36.82.44:4882 , piece_id=46O66XTAJO5M5EIN75SIZ7HAFK6C3P5VNH2R3BN3HXRLBPHQOV2Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: <nil>
Jul 9 00:38:03.607853389 (375h27m3.491267022s) [seg_pos={0 0}, upload=300223]: Segment reached optimal threshold of 14 pieces.
Jul 9 00:38:03.608006922 (375h27m3.491420552s) [noise=true , address=104.36.82.19:4427 , piece_id=LGLX2VGSEE6SVNRZSKX46A5WGT3SLKH2EO6VM3SYJXRZX5V5GP6Q, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608045600 (375h27m3.491459233s) [noise=true , address=104.36.82.4:4809 , piece_id=WYLOHB5UOZWNLSPDSZ2GNFVQGG5HD4OIKKZGPF5ZFMCHU45MGNYQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17LrYeCkrPWdiL9gGZ568PvfFct6cLVRQE4Qa92ykLgqXSfBL9): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608087309 (375h27m3.491500937s) [noise=true , address=104.36.82.22:4540 , piece_id=NR7IGPRLKNCRVYDVMGTMAPHHAV5W7VHEMCUDZ3C7MX372WH765ZQ, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RY6XGL5MJYc2idL6Np7DS3U9pCQGDhuEEqguY8AgjasjE4hiZ): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608191697 (375h27m3.491605324s) [noise=true , address=104.36.82.36:4631 , piece_id=WDDGALSSDOMMI2SRRAXKWD3SD7WI7PCA4L33Q2WP3IZ7UTMH2R4A, seg_pos={0 0}, upload=300223]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T6FqqMunZ46QoxhHVWsn8NinZGWv1Yo7rNHSiK6s9hhuQtS6Y): protocol: expected piece hash; context canceled; EOF
Jul 9 00:38:03.608213412 (375h27m3.49162703s) [seg_pos={0 0}, upload=300223]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 00:38:03.608310462 (375h27m3.49172408s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:38:03.608316810 (375h27m3.491730438s) [upload=300223]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:03.761052568 (375h27m3.6444662s) [upload=300223]: Done uploading segments...
Jul 9 00:38:03.761076757 (375h27m3.644490378s) [upload=300223]: Upload finished. err: <nil>
Duration: 19.67491252s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
11 out of 60 is failed (threshold=3) [.XX...X..............X....X..X......X.............XX.X.X....] duration:
3.133364544s
FAIL (2025-07-09 00:55:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.133423525s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
13 out of 60 is failed (threshold=3) [.................X...........XX.X..X..X..XX.X...........XXXX] duration:
1m0.001241236s
FAIL (2025-07-09 00:55:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001269343s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................................X..X.....................] duration:
2.953137054s
FAIL (2025-07-09 00:55:27 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 10)
======LOGS======
entCommit
Jul 9 00:38:54.141867066 (375h27m54.025280684s) [upload=300248]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:38:54.286588243 (375h27m54.170001881s) [upload=300248]: Done uploading segments...
Jul 9 00:38:54.286633475 (375h27m54.170047105s) [upload=300248]: Upload finished. err: <nil>
Jul 9 00:39:49.853458099 (375h28m49.736871729s) [upload=300262]: Starting upload
Jul 9 00:39:49.853564556 (375h28m49.736978174s) [upload=300262]: Done starting upload
Jul 9 00:39:49.853754759 (375h28m49.737168394s) [upload=300262]: Uploading segments...
Jul 9 00:39:49.854072231 (375h28m49.737485857s) [upload=300262]: Got next segment. Inline: false
Jul 9 00:39:49.854139888 (375h28m49.737553515s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:39:49.854145670 (375h28m49.737559297s) [upload=300262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:39:49.964717413 (375h28m49.848131039s) [seg_pos={0 0}, upload=300262]: Begin upload segment...
Jul 9 00:39:49.964772224 (375h28m49.848185839s) [seg_pos={0 0}, upload=300262]: Done begin upload segment.
Jul 9 00:39:49.964909767 (375h28m49.848323393s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965422318 (375h28m49.848835947s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.965585955 (375h28m49.848999581s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966217091 (375h28m49.849630727s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.966292916 (375h28m49.849706542s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967093655 (375h28m49.85050729s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967169721 (375h28m49.850583348s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.967940005 (375h28m49.851353634s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.968588971 (375h28m49.8520026s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969080450 (375h28m49.852494085s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.969563775 (375h28m49.852977409s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.970736354 (375h28m49.854149991s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.971229734 (375h28m49.85464336s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.972867679 (375h28m49.856281317s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.978267895 (375h28m49.861681529s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:49.986076032 (375h28m49.869489662s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Uploading piece...
Jul 9 00:39:50.595435555 (375h28m50.478849173s) [upload=300262]: Next returned nil segment
Jul 9 00:39:50.595455642 (375h28m50.478869272s) [upload=300262]: Waiting for error group managing segments...
Jul 9 00:39:50.730628340 (375h28m50.614041956s) [noise=true , address=45.91.116.4:4938 , piece_id=OTSXKKNFPSNIANVVIRPDRQEFFDWI27D2JNDGPYDI3QIECWKNJVVA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.731179161 (375h28m50.614592788s) [noise=true , address=45.91.116.13:4153 , piece_id=EBVIZJFQDVIOWT7LM24W5JL36AS2QIOOL4BE7U2MWAK4NUMS763Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.732951876 (375h28m50.616365493s) [noise=true , address=45.91.116.11:4119 , piece_id=VVCKJQ525IGLIAUI2QHK6IVTF6CUSVRLATUN52QMBKLJYSZCTENQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.738403976 (375h28m50.621817592s) [noise=true , address=45.91.116.13:4154 , piece_id=L6UICK5NFWW7X7BW7JM6PPAWRQL5RUXV7UDAOSGRDPMR5MILCHEA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739011956 (375h28m50.622425573s) [noise=true , address=45.91.116.8:4045 , piece_id=QJNLPTWM7SKNXSVXRPVYXQJDI23YZ5I4B4SQRYGYWOFW27MWWMWA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.739658988 (375h28m50.623072606s) [noise=true , address=45.91.116.7:4010 , piece_id=LRVYKKDEGXKZJKVX3AFCBSA4MFQWIOZROXBHEY4RC2TFBWMU4Y7Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:50.766298764 (375h28m50.649712392s) [noise=true , address=77.90.31.21:4879 , piece_id=UUSCN7CIUVOMFWISVIIPUCP55VVLJD73FOE44KUWGDX4GCFHB7FQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.754942777 (375h28m51.638356391s) [noise=true , address=104.36.82.16:4293 , piece_id=C4B5T24WIBRTLCYICMPMYJCVSOPAP3VY22AHWUKGGU5T4A4DMWRQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:39:51.990915358 (375h28m51.87432899s) [noise=true , address=104.36.82.16:4295 , piece_id=H277ZJJ3PJ4JKGGXGSCQMCIFBS6RH4EOKIR6G5GSDSNUAYSW6RZA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:13.196278807 (375h29m13.079692437s) [noise=true , address=104.36.82.5:4837 , piece_id=GFLNL2X2E74H4EPHFZZMQ7QELCNUZ45L7QYLRWJXGVXZT2U3KGFA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: <nil>
Jul 9 00:40:19.575930850 (375h29m19.459344476s) [noise=true , address=104.36.82.15:4259 , piece_id=SAFMGMQWVIL2ONP6KBUHVBVRNL3LT7WUDH24SGNWB745D62X6QXQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576005813 (375h29m19.459419439s) [noise=true , address=104.36.82.2:4730 , piece_id=5XR73Z34ZYCGLRYV2ZQWCNCNKX5I7P5MYXT254ZVEHSVPIZ5RGCA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576051734 (375h29m19.459465361s) [noise=true , address=104.36.82.4:4804 , piece_id=5G2JOP4O2EHZVIDIOJBELVH54XPRHSQFXEDKNSD4K4CO4AJYJKFQ, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576066765 (375h29m19.459480398s) [noise=true , address=104.36.82.13:4185 , piece_id=CFDK2QCRSTTHZ4IDXF4Q346AFBDG2KTWOLXAKZX4JWMX7JR754CA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:40:19.576255242 (375h29m19.459668869s) [noise=true , address=104.36.82.3:4767 , piece_id=XT74I2XQGL56SLXVHN6TLIXLS7LW3VE5H45VE5ZDCTOSBIV5AX2Q, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38824->104.36.82.3:4767: use of closed network connection
Jul 9 00:40:19.576293041 (375h29m19.459706667s) [noise=true , address=104.36.82.8:4971 , piece_id=ZGDLV2DBTJODCRLQT3G6ZU3SADRENQXMTYNPNX2NMZBAXOD4SWNA, seg_pos={0 0}, upload=300262]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37934->104.36.82.8:4971: use of closed network connection
Jul 9 00:40:19.576316511 (375h29m19.45973014s) [seg_pos={0 0}, upload=300262]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil>
Jul 9 00:40:19.685159343 (375h29m19.568572971s) [upload=300262]: Done uploading segments...
Jul 9 00:40:19.685187481 (375h29m19.568601108s) [upload=300262]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10)
Duration: 2.953164936s
core
uplink
us-select-1
|