Orbiter History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 443.672526ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:54:33 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 443.704428ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 469.968239ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:54:16 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 469.996594ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 428.101473ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:53:40 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 428.127614ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 799.980317ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:52:59 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 800.005583ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 564.971036ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:52:39 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 565.005568ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 440.223163ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:52:00 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 440.255388ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 431.91836ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:51:45 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 431.942371ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 429.427642ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:51:24 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 429.447651ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 395.929207ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:50:22 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 395.979403ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 867.952292ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:50:06 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 867.982531ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 433.345438ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:49:36 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 433.37129ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 442.177511ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:49:04 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 442.231409ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 449.123339ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:48:17 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 449.145589ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 427.133023ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:47:33 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 427.157254ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 432.972137ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:46:42 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 432.998308ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 398.784289ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:45:34 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 398.832568ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 492.159979ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:44:45 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 492.187854ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 577.341709ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:44:00 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 577.414843ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 467.501701ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:43:13 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 467.530366ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 789.528178ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:41:59 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 789.614758ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 493.155877ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:41:34 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 493.190917ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 477.810753ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:40:24 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 477.842919ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 781.770072ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:40:04 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 781.801969ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 433.280487ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:39:15 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 433.400492ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 448.00628ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:38:23 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 448.037015ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 428.982474ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:37:16 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 429.011899ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 510.845531ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:36:22 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 510.870251ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 412.14688ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:35:33 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 412.178546ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 613.939442ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:35:03 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 613.973154ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 428.8316ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:34:39 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 428.863782ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 429.567323ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:33:47 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 429.593052ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 412.719112ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:32:32 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 412.743619ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 430.145883ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:31:43 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 430.176442ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 448.717216ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:30:37 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 448.736394ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 507.607939ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:30:18 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 507.631005ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 460.507191ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:29:40 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 460.533098ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 428.324003ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:28:32 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 428.350799ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 468.017983ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:27:44 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 468.049375ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 531.438726ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:27:00 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 531.473195ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 468.760156ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:26:37 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 468.788803ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 420.716868ms trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:26:09 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 420.735783ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.003312417s trace: https://jaeger.dev.storj.io/trace/6f03109d869521d FAIL (2025-10-31 19:26:08 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 67.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956053430 (31h44m11.657645883s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.956513382 (31h44m11.658105833s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957203079 (31h44m11.658795537s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.957810763 (31h44m11.65940323s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.958490731 (31h44m11.660083188s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.959102151 (31h44m11.660694608s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960053970 (31h44m11.661646453s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.960748416 (31h44m11.66234087s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.961426224 (31h44m11.663018685s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:08.962221532 (31h44m11.663813994s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Uploading piece... Oct 31 19:25:09.089405055 (31h44m11.790997516s) [upload=27918]: Next returned nil segment Oct 31 19:25:09.089426932 (31h44m11.791019394s) [upload=27918]: Waiting for error group managing segments... Oct 31 19:25:09.094025091 (31h44m11.795617543s) [noise=true , address=89.149.201.147:4008 , piece_id=S7N4S3EQEKJDZGX6R4AIJTJFJCH7KR4GIADESGIQJEPKC73BNW3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094079311 (31h44m11.795671763s) [noise=true , address=81.171.14.70:4040 , piece_id=Q4QVVAIAYAVEI5GFW6NECJONEGPKQUGLS5SJFZMSG3RTL53TUDDA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.094800800 (31h44m11.796393252s) [noise=true , address=89.149.201.147:4023 , piece_id=XVYPIYOLHLXN6MF67SJ3MZGZFY6CULBZ6UZDHH6LNSJE4HBGDLQQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.098897378 (31h44m11.800489835s) [noise=true , address=185.167.204.205:4024 , piece_id=MYBXNEJNPW5R2WCZJWYVEUBCLRSSXXNBJWNMOWATL3YYWZF742XQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099395156 (31h44m11.800987608s) [noise=true , address=185.167.204.205:4022 , piece_id=4E2JGAHF23TWJWVYBLE56MGMTMRLWEH33RJYPEHQHAWK3J7CEVXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.099863521 (31h44m11.801455977s) [noise=true , address=185.167.204.206:4067 , piece_id=QUYEKUSPGTKCRIHFXUCY4X3L6SAODNN6BADXL2EXWFY66Z5AX7PA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100294404 (31h44m11.801886858s) [noise=true , address=81.171.14.70:4028 , piece_id=ZIEOL6VSTZ4VTQT57WGM6WTX7QLUGYA72LOIPPFR6KTIVMJ7UYMA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.100633412 (31h44m11.802225867s) [noise=true , address=185.167.204.206:4056 , piece_id=PJRD7FGPTEQYYUCT745F7ZI52SZUIFU5TFNEIKVJCN6TKS3TUEHA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.101434913 (31h44m11.803027366s) [noise=true , address=185.167.204.206:4058 , piece_id=MBRFSYBYMWH3SBBDW42YNWXYK37SYHFT2IOCKJS7BMFPMIBKHQFQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102148306 (31h44m11.80374076s) [noise=true , address=81.171.14.70:4032 , piece_id=ESLRC2JVPYSPZQVBYCFVVLRYJZVPTVM5GJJQXACR2CWWHMNA5B5A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102516901 (31h44m11.804109353s) [noise=true , address=81.171.14.70:4027 , piece_id=ZZB75QQLI6JIPRALC3MJLK6LJ3X6GL3UDA24IGRV6UHUCO65GQBQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.102894823 (31h44m11.804487275s) [noise=true , address=185.167.204.205:4006 , piece_id=K25XGQSKX3FDM2KHDNHTRWXNTLMCZDHIZBZSW7EHZSKDCXABFGCA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.106832421 (31h44m11.808424907s) [noise=true , address=185.167.204.206:4047 , piece_id=5QFDMPGTEVZNDII23JQ33TY6SOCJO5ZHRE7TWQWV5G22DZNLDZUQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.107129826 (31h44m11.808722281s) [noise=true , address=185.167.204.205:4021 , piece_id=SEBXV4JSZNQO7LOH3WSBXA2ECKFAI4S3VJRAVHTDPYKH3N62JAKA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.109066332 (31h44m11.810658789s) [noise=true , address=81.171.14.70:4047 , piece_id=LO3PBEFIPOWIRLUTA2X5HNGDFUEVS7NKDVNWH2CSJGHZBDNZYX7A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.112900829 (31h44m11.814493287s) [noise=true , address=185.167.204.206:4057 , piece_id=BAJK6JMS3TDIMHOT6NF2PBIFYHJC34M3VU4E2EJ6G4XJGUGU6OTA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.114375084 (31h44m11.815967584s) [noise=true , address=89.149.201.147:4020 , piece_id=LXTZSAAU3DIQMVSM2XCJSRJU535YFMU622XAYR2WMRHC6ZLNON4A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.117391042 (31h44m11.818983505s) [noise=true , address=81.171.14.70:4043 , piece_id=E7H7DCWJ4GRRXNVGEJOY55QKZKGFE3CXDSBGO2FMPXEPMBY6AWSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.118935135 (31h44m11.820527591s) [noise=true , address=89.149.201.147:4007 , piece_id=47PWRIA24G4AGRTKTPLQ6PLCLZJQFVZJ4HRSATC3YHVWTJADMB3A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120275173 (31h44m11.821867629s) [noise=true , address=81.171.14.70:4026 , piece_id=MT5VEPQDGPEXUO3QT634PU2IQU47LM5ZRX4SJXVTPO2WBR5BL2CQ, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.120987615 (31h44m11.822580075s) [noise=true , address=89.149.201.147:4012 , piece_id=3HKVP2MMAVJOJCNLMOTTDDSCPA4QMXIK4FJ24562LFUYTIR65A6A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.121669813 (31h44m11.823262255s) [noise=true , address=81.171.14.70:4046 , piece_id=PMQPN35FZBRAK7DDZHJBWR5BWB3VJRCCCZXUWATN6DG3XA72WGSA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.122378445 (31h44m11.823970895s) [noise=true , address=185.167.204.206:4050 , piece_id=5OKBYIXVWQWKXQXCQYFY74BA7FI3Z46QOTSJRZOERC63A3NNOHXA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.123052255 (31h44m11.824644705s) [noise=true , address=81.171.14.70:4029 , piece_id=XHWCZU2MD2YL6QW4O24T6F6UBZEIFWRL3YFYYZWQAO3LEUTUD4JA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126273483 (31h44m11.827865933s) [noise=true , address=89.149.201.147:4017 , piece_id=XV3VZFXYHIHHF4QLFVBJVW3RCEFAY7DJDYQ2B2LLK3V54LQMV77A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126589016 (31h44m11.828181478s) [noise=true , address=89.149.201.147:4003 , piece_id=IVZNQRKCAT4G7KPBSFURS6H43VSGOTPMSYEAQ42YX5QVLH7TBMUA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: <nil> Oct 31 19:25:09.126653182 (31h44m11.828245644s) [seg_pos={0 0}, upload=27918]: Segment reached optimal threshold of 26 pieces. Oct 31 19:25:09.127135459 (31h44m11.828727924s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127151429 (31h44m11.82874389s) [noise=true , address=89.149.201.147:4021 , piece_id=NQ3UH23HT5V34KM5RNMMYVJCMQNESWNI5W2T2COR26U3DJVP64GA, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127365200 (31h44m11.828957662s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Oct 31 19:25:09.127375134 (31h44m11.828967596s) [noise=true , address=89.149.201.147:4002 , piece_id=RQXIWZCSEJIH4CS6NAUHNQ6MC2TMGJFE4RB5EZIREZDRA4Q7R54A, seg_pos={0 0}, upload=27918]: Long tail cancellation due to optimal threshold Oct 31 19:25:09.127405694 (31h44m11.828998146s) [seg_pos={0 0}, upload=27918]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 31 19:25:09.127517223 (31h44m11.829109692s) [upload=27918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 31 19:25:09.127523946 (31h44m11.829116408s) [upload=27918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 31 19:25:09.163266197 (31h44m11.864858653s) [upload=27918]: Done uploading segments... Oct 31 19:25:09.163420243 (31h44m11.865012694s) [upload=27918]: Upload finished. err: <nil> Duration: 1m0.00335782s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [X...........................................................] duration: 1.63217197s trace: https://jaeger.dev.storj.io/trace/4220b99ce2cf1806 FAIL (2025-10-31 19:08:23 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.632258382s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [XX..........................................................] duration: 1.709567792s trace: https://jaeger.dev.storj.io/trace/4220b99ce2cf1806 FAIL (2025-10-31 19:07:17 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.709588885s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.XX.........................................................] duration: 1.60519103s trace: https://jaeger.dev.storj.io/trace/4220b99ce2cf1806 FAIL (2025-10-31 19:06:25 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.605218704s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..XX........................................................] duration: 1.701831505s trace: https://jaeger.dev.storj.io/trace/4220b99ce2cf1806 FAIL (2025-10-31 19:05:15 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.701871139s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...XX.......................................................] duration: 1.688865423s trace: https://jaeger.dev.storj.io/trace/4220b99ce2cf1806 FAIL (2025-10-31 19:05:00 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.688950356s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [....XX......................................................] duration: 1.745656246s trace: https://jaeger.dev.storj.io/trace/4220b99ce2cf1806 FAIL (2025-10-31 19:04:16 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.745677721s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.....XX.....................................................] duration: 1.617549505s trace: https://jaeger.dev.storj.io/trace/4220b99ce2cf1806 FAIL (2025-10-31 19:03:28 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.617611333s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [......XX....................................................] duration: 1.627490596s trace: https://jaeger.dev.storj.io/trace/4220b99ce2cf1806 FAIL (2025-10-31 19:02:45 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.627521538s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)