Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................X...X........................] duration: 2.270926825s FAIL (2025-10-07 10:32:17 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.27097188s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.443572468s FAIL (2025-10-07 10:31:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.443614018s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................X...X.......................] duration: 2.040546722s FAIL (2025-10-07 10:31:43 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.040574257s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.551181584s FAIL (2025-10-07 10:31:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.551214061s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................X...X......................] duration: 2.245255141s FAIL (2025-10-07 10:30:48 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.245295521s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.48556794s FAIL (2025-10-07 10:30:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.485601286s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.527212616s FAIL (2025-10-07 10:30:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.527252719s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................X...X.....................] duration: 2.18740952s FAIL (2025-10-07 10:29:52 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.18744594s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.377005291s FAIL (2025-10-07 10:29:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.377035833s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................X...X....................] duration: 2.04606725s FAIL (2025-10-07 10:28:58 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.046103147s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.506967093s FAIL (2025-10-07 10:28:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.506994128s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................X...X...................] duration: 2.197733148s FAIL (2025-10-07 10:28:31 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.19776969s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X...X..................] duration: 2.195543733s FAIL (2025-10-07 10:27:47 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.195574689s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.416396383s FAIL (2025-10-07 10:27:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.416460104s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................X...X.................] duration: 2.146718465s FAIL (2025-10-07 10:27:07 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.146752307s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.329528124s FAIL (2025-10-07 10:26:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.329563107s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X...X................] duration: 2.24630336s FAIL (2025-10-07 10:26:32 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.246329769s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.430693763s FAIL (2025-10-07 10:26:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.430725449s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................X...X...............] duration: 2.205937387s FAIL (2025-10-07 10:25:33 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.205974085s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.513849752s FAIL (2025-10-07 10:25:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.513881823s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X...X..............] duration: 1.801483971s FAIL (2025-10-07 10:24:58 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.801516681s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.625435864s FAIL (2025-10-07 10:24:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.625477554s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.454517191s FAIL (2025-10-07 10:23:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.454565127s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X...X.............] duration: 2.075522611s FAIL (2025-10-07 10:23:44 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.075557614s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.371539707s FAIL (2025-10-07 10:23:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.371586987s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X...X............] duration: 2.110880886s FAIL (2025-10-07 10:22:54 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.110912303s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.501177781s FAIL (2025-10-07 10:22:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.501204297s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.443487642s FAIL (2025-10-07 10:22:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.44356423s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................................X...X...........] duration: 1.957663978s FAIL (2025-10-07 10:21:54 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.957699129s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.427600177s FAIL (2025-10-07 10:21:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.427718925s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.450192509s FAIL (2025-10-07 10:20:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.450221402s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X...X..........] duration: 2.318985965s FAIL (2025-10-07 10:20:41 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.319017573s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.351147144s FAIL (2025-10-07 10:20:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.351188033s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X...X.........] duration: 2.112081541s FAIL (2025-10-07 10:19:41 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.112143603s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................................X...X........] duration: 2.117227304s FAIL (2025-10-07 10:18:56 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.117273322s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.543346822s FAIL (2025-10-07 10:18:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.543401967s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.448767785s FAIL (2025-10-07 10:18:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.448794625s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................................X...X.......] duration: 1.904971832s FAIL (2025-10-07 10:17:47 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.905011233s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................................X...X......] duration: 2.173811815s FAIL (2025-10-07 10:17:13 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.173851123s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.572985381s FAIL (2025-10-07 10:17:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.573016798s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................................X...X.....] duration: 2.201477293s FAIL (2025-10-07 10:16:32 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.201518533s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.353096028s FAIL (2025-10-07 10:16:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.353136722s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................X...X....] duration: 2.496488187s FAIL (2025-10-07 10:15:50 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.496524293s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.367747841s FAIL (2025-10-07 10:15:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.36777926s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................X...X...] duration: 2.210875043s FAIL (2025-10-07 10:14:56 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.210922203s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.389024797s FAIL (2025-10-07 10:14:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.389145879s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................X...X..] duration: 2.186670129s FAIL (2025-10-07 10:14:14 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.186710475s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 22.321153678s FAIL (2025-10-07 10:13:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 22.321177734s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................X...X.] duration: 22.033912386s FAIL (2025-10-07 10:13:35 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 22.033946879s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 22.537859577s FAIL (2025-10-07 10:13:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 22.537902247s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)