Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................X.............XX....] duration: 2.481298077s trace: https://jaeger.dev.storj.io/trace/c055a066e80622b FAIL (2025-11-10 01:33:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 10 01:29:50.180949372 (253h48m52.882541824s) [upload=223211]: Starting upload Nov 10 01:29:50.181032473 (253h48m52.882624934s) [upload=223211]: Done starting upload Nov 10 01:29:50.181156682 (253h48m52.882749125s) [upload=223211]: Uploading segments... Nov 10 01:29:50.181465875 (253h48m52.883058315s) [upload=223211]: Got next segment. Inline: false Nov 10 01:29:50.181499404 (253h48m52.883091854s) [upload=223211]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 10 01:29:50.181504566 (253h48m52.883097006s) [upload=223211]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 10 01:29:50.288289364 (253h48m52.989881813s) [seg_pos={0 0}, upload=223211]: Begin upload segment... Nov 10 01:29:50.288363675 (253h48m52.989956127s) [seg_pos={0 0}, upload=223211]: Done begin upload segment. Nov 10 01:29:50.288400256 (253h48m52.989992693s) [noise=true , address=145.79.227.21:4474 , piece_id=ITRP3SCJJ7JRC2SW54DV7CWXTVZHVJDGFSPA2VDR45CNX6UAAG3Q, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.288416601 (253h48m52.990009056s) [noise=true , address=145.79.227.20:4438 , piece_id=X26D2AKZ2CXYBKKBTFP6TB3MWL5LSW4JKGGZN43CIU3AU5IN3JWQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.288921316 (253h48m52.990513768s) [noise=true , address=77.90.31.10:4520 , piece_id=FWC3GNZIQHRM3TBWUI42BYKLCNZ3LMYZNOMAFZEGEZS2LDWMHYBA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.289398939 (253h48m52.99099142s) [noise=true , address=145.79.227.13:4803 , piece_id=ZPOECQBGXDFAM5MYW5CM73XRS56R44CG4UD74SCHBQEVTCGPEBHQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.289886286 (253h48m52.991478737s) [noise=true , address=145.79.227.10:4403 , piece_id=DKPD6QFB4VDYUTF46OQZIYDLXA6XTSWDOMZDI2O35RIWIQSYWUOQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.290023453 (253h48m52.991615891s) [noise=true , address=145.79.227.26:4655 , piece_id=SYKVYBTUO2M3VZ34MHIRAMEGVGBM6OZWUYQ46C35VSZYETX2FEZQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.290402347 (253h48m52.991994805s) [noise=true , address=145.79.227.15:4876 , piece_id=THTJQZY5CJFDOZ64OI7RD3VP5EEOKHVT4T6MS5G3Z5AEQO2AVVBQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.290460076 (253h48m52.992052527s) [noise=true , address=77.90.31.21:4878 , piece_id=LRYUY6I45PIRGCLW6TH5DT4NZ4BCG7FSWTQ5KNQV6YDWQVVCRPCA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.290860429 (253h48m52.992452885s) [noise=true , address=145.79.227.14:4837 , piece_id=6UJFDJPFALMG7HYK5DBZU7VV4GU27ROOFXOLG7WVSRQE6JBZMYKA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.291265673 (253h48m52.992858124s) [noise=true , address=145.79.227.30:4762 , piece_id=M2V6D3VWJMURCFRK5YFNVMAO2AIOBM2GWSL6OBZJE5HUY77IJLFQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.291688246 (253h48m52.993280698s) [noise=true , address=145.79.227.28:4726 , piece_id=53QO5GLB3GC64YZ3Z6ALE3O4KSPDNHSDE7QKWMX7SPHYWXLCDBTA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.292140757 (253h48m52.993733216s) [noise=true , address=145.79.227.32:4835 , piece_id=SQ237HBZSYOFGIJHNRBJ3W4R5YVMQ6QLAMCJRSMLMQ4E76RFVFDA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.292895942 (253h48m52.994488396s) [noise=true , address=145.79.227.8:4331 , piece_id=AHJM74RULWU2OLNPHSARMLJWUB5L343PESDONTTWECSQB7SI4Y2A, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.293619050 (253h48m52.995211507s) [noise=true , address=145.79.227.16:4911 , piece_id=UIG36LK3PQNRTF5F47JI6UM5ITPLPPQNB3R4ORBHSLCLAST7NAPA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.294091314 (253h48m52.995683765s) [noise=true , address=77.90.31.15:4699 , piece_id=ZTT7EEHL2PBGXEEDZ5GOSHHCCQLSSIAMHK7AUDDFPCDS75FYQLOA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.294213128 (253h48m52.995805644s) [noise=true , address=145.79.227.6:4258 , piece_id=ADOYO4XPHEUPCBE7RR6IM4WIQXCAFSAYC7OVY56PD62Y43HDWTOA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.898598865 (253h48m53.600191317s) [upload=223211]: Next returned nil segment Nov 10 01:29:50.898616587 (253h48m53.600209045s) [upload=223211]: Waiting for error group managing segments... Nov 10 01:29:51.030451931 (253h48m53.732044379s) [noise=true , address=77.90.31.10:4520 , piece_id=FWC3GNZIQHRM3TBWUI42BYKLCNZ3LMYZNOMAFZEGEZS2LDWMHYBA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:51.051603630 (253h48m53.753196085s) [noise=true , address=77.90.31.15:4699 , piece_id=ZTT7EEHL2PBGXEEDZ5GOSHHCCQLSSIAMHK7AUDDFPCDS75FYQLOA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:51.051711369 (253h48m53.75330382s) [noise=true , address=77.90.31.21:4878 , piece_id=LRYUY6I45PIRGCLW6TH5DT4NZ4BCG7FSWTQ5KNQV6YDWQVVCRPCA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:51.532787951 (253h48m54.234380399s) [noise=true , address=145.79.227.15:4876 , piece_id=THTJQZY5CJFDOZ64OI7RD3VP5EEOKHVT4T6MS5G3Z5AEQO2AVVBQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:51.765229559 (253h48m54.466822016s) [noise=true , address=145.79.227.26:4655 , piece_id=SYKVYBTUO2M3VZ34MHIRAMEGVGBM6OZWUYQ46C35VSZYETX2FEZQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:52.009059018 (253h48m54.710651469s) [noise=true , address=145.79.227.6:4258 , piece_id=ADOYO4XPHEUPCBE7RR6IM4WIQXCAFSAYC7OVY56PD62Y43HDWTOA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:52.009456880 (253h48m54.711049319s) [noise=true , address=145.79.227.28:4726 , piece_id=53QO5GLB3GC64YZ3Z6ALE3O4KSPDNHSDE7QKWMX7SPHYWXLCDBTA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:52.249419947 (253h48m54.9510124s) [noise=true , address=145.79.227.13:4803 , piece_id=ZPOECQBGXDFAM5MYW5CM73XRS56R44CG4UD74SCHBQEVTCGPEBHQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:52.747418960 (253h48m55.44901141s) [noise=true , address=145.79.227.20:4438 , piece_id=X26D2AKZ2CXYBKKBTFP6TB3MWL5LSW4JKGGZN43CIU3AU5IN3JWQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:53.507082237 (253h48m56.208674693s) [noise=true , address=145.79.227.30:4762 , piece_id=M2V6D3VWJMURCFRK5YFNVMAO2AIOBM2GWSL6OBZJE5HUY77IJLFQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:53.566714417 (253h48m56.268306873s) [noise=true , address=145.79.227.32:4835 , piece_id=SQ237HBZSYOFGIJHNRBJ3W4R5YVMQ6QLAMCJRSMLMQ4E76RFVFDA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:30:19.906744987 (253h49m22.608337469s) [noise=true , address=145.79.227.10:4403 , piece_id=DKPD6QFB4VDYUTF46OQZIYDLXA6XTSWDOMZDI2O35RIWIQSYWUOQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 139.59.144.22:35544->145.79.227.10:4403: use of closed network connection; write tcp 139.59.144.22:35544->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35544->145.79.227.10:4403: use of closed network connection Nov 10 01:30:19.906915176 (253h49m22.608507655s) [noise=true , address=145.79.227.8:4331 , piece_id=AHJM74RULWU2OLNPHSARMLJWUB5L343PESDONTTWECSQB7SI4Y2A, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: write tcp 139.59.144.22:57352->145.79.227.8:4331: use of closed network connection; write tcp 139.59.144.22:57352->145.79.227.8:4331: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57352->145.79.227.8:4331: use of closed network connection Nov 10 01:30:19.907148249 (253h49m22.608740766s) [noise=true , address=145.79.227.14:4837 , piece_id=6UJFDJPFALMG7HYK5DBZU7VV4GU27ROOFXOLG7WVSRQE6JBZMYKA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: read tcp 139.59.144.22:48446->145.79.227.14:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:30:19.907253647 (253h49m22.608846101s) [noise=true , address=145.79.227.16:4911 , piece_id=UIG36LK3PQNRTF5F47JI6UM5ITPLPPQNB3R4ORBHSLCLAST7NAPA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 139.59.144.22:38920->145.79.227.16:4911: use of closed network connection; write tcp 139.59.144.22:38920->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38920->145.79.227.16:4911: use of closed network connection Nov 10 01:30:19.907481275 (253h49m22.609073737s) [noise=true , address=145.79.227.21:4474 , piece_id=ITRP3SCJJ7JRC2SW54DV7CWXTVZHVJDGFSPA2VDR45CNX6UAAG3Q, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: context deadline exceeded; context deadline exceeded Nov 10 01:30:19.907515860 (253h49m22.609108309s) [seg_pos={0 0}, upload=223211]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 10 01:30:20.026141819 (253h49m22.727734268s) [upload=223211]: Done uploading segments... Nov 10 01:30:20.026177189 (253h49m22.727769638s) [upload=223211]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.48133343s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................X.............XX...] duration: 2.330072065s trace: https://jaeger.dev.storj.io/trace/c055a066e80622b FAIL (2025-11-10 01:32:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 10 01:29:50.180949372 (253h48m52.882541824s) [upload=223211]: Starting upload Nov 10 01:29:50.181032473 (253h48m52.882624934s) [upload=223211]: Done starting upload Nov 10 01:29:50.181156682 (253h48m52.882749125s) [upload=223211]: Uploading segments... Nov 10 01:29:50.181465875 (253h48m52.883058315s) [upload=223211]: Got next segment. Inline: false Nov 10 01:29:50.181499404 (253h48m52.883091854s) [upload=223211]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 10 01:29:50.181504566 (253h48m52.883097006s) [upload=223211]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 10 01:29:50.288289364 (253h48m52.989881813s) [seg_pos={0 0}, upload=223211]: Begin upload segment... Nov 10 01:29:50.288363675 (253h48m52.989956127s) [seg_pos={0 0}, upload=223211]: Done begin upload segment. Nov 10 01:29:50.288400256 (253h48m52.989992693s) [noise=true , address=145.79.227.21:4474 , piece_id=ITRP3SCJJ7JRC2SW54DV7CWXTVZHVJDGFSPA2VDR45CNX6UAAG3Q, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.288416601 (253h48m52.990009056s) [noise=true , address=145.79.227.20:4438 , piece_id=X26D2AKZ2CXYBKKBTFP6TB3MWL5LSW4JKGGZN43CIU3AU5IN3JWQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.288921316 (253h48m52.990513768s) [noise=true , address=77.90.31.10:4520 , piece_id=FWC3GNZIQHRM3TBWUI42BYKLCNZ3LMYZNOMAFZEGEZS2LDWMHYBA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.289398939 (253h48m52.99099142s) [noise=true , address=145.79.227.13:4803 , piece_id=ZPOECQBGXDFAM5MYW5CM73XRS56R44CG4UD74SCHBQEVTCGPEBHQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.289886286 (253h48m52.991478737s) [noise=true , address=145.79.227.10:4403 , piece_id=DKPD6QFB4VDYUTF46OQZIYDLXA6XTSWDOMZDI2O35RIWIQSYWUOQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.290023453 (253h48m52.991615891s) [noise=true , address=145.79.227.26:4655 , piece_id=SYKVYBTUO2M3VZ34MHIRAMEGVGBM6OZWUYQ46C35VSZYETX2FEZQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.290402347 (253h48m52.991994805s) [noise=true , address=145.79.227.15:4876 , piece_id=THTJQZY5CJFDOZ64OI7RD3VP5EEOKHVT4T6MS5G3Z5AEQO2AVVBQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.290460076 (253h48m52.992052527s) [noise=true , address=77.90.31.21:4878 , piece_id=LRYUY6I45PIRGCLW6TH5DT4NZ4BCG7FSWTQ5KNQV6YDWQVVCRPCA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.290860429 (253h48m52.992452885s) [noise=true , address=145.79.227.14:4837 , piece_id=6UJFDJPFALMG7HYK5DBZU7VV4GU27ROOFXOLG7WVSRQE6JBZMYKA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.291265673 (253h48m52.992858124s) [noise=true , address=145.79.227.30:4762 , piece_id=M2V6D3VWJMURCFRK5YFNVMAO2AIOBM2GWSL6OBZJE5HUY77IJLFQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.291688246 (253h48m52.993280698s) [noise=true , address=145.79.227.28:4726 , piece_id=53QO5GLB3GC64YZ3Z6ALE3O4KSPDNHSDE7QKWMX7SPHYWXLCDBTA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.292140757 (253h48m52.993733216s) [noise=true , address=145.79.227.32:4835 , piece_id=SQ237HBZSYOFGIJHNRBJ3W4R5YVMQ6QLAMCJRSMLMQ4E76RFVFDA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.292895942 (253h48m52.994488396s) [noise=true , address=145.79.227.8:4331 , piece_id=AHJM74RULWU2OLNPHSARMLJWUB5L343PESDONTTWECSQB7SI4Y2A, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.293619050 (253h48m52.995211507s) [noise=true , address=145.79.227.16:4911 , piece_id=UIG36LK3PQNRTF5F47JI6UM5ITPLPPQNB3R4ORBHSLCLAST7NAPA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.294091314 (253h48m52.995683765s) [noise=true , address=77.90.31.15:4699 , piece_id=ZTT7EEHL2PBGXEEDZ5GOSHHCCQLSSIAMHK7AUDDFPCDS75FYQLOA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.294213128 (253h48m52.995805644s) [noise=true , address=145.79.227.6:4258 , piece_id=ADOYO4XPHEUPCBE7RR6IM4WIQXCAFSAYC7OVY56PD62Y43HDWTOA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.898598865 (253h48m53.600191317s) [upload=223211]: Next returned nil segment Nov 10 01:29:50.898616587 (253h48m53.600209045s) [upload=223211]: Waiting for error group managing segments... Nov 10 01:29:51.030451931 (253h48m53.732044379s) [noise=true , address=77.90.31.10:4520 , piece_id=FWC3GNZIQHRM3TBWUI42BYKLCNZ3LMYZNOMAFZEGEZS2LDWMHYBA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:51.051603630 (253h48m53.753196085s) [noise=true , address=77.90.31.15:4699 , piece_id=ZTT7EEHL2PBGXEEDZ5GOSHHCCQLSSIAMHK7AUDDFPCDS75FYQLOA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:51.051711369 (253h48m53.75330382s) [noise=true , address=77.90.31.21:4878 , piece_id=LRYUY6I45PIRGCLW6TH5DT4NZ4BCG7FSWTQ5KNQV6YDWQVVCRPCA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:51.532787951 (253h48m54.234380399s) [noise=true , address=145.79.227.15:4876 , piece_id=THTJQZY5CJFDOZ64OI7RD3VP5EEOKHVT4T6MS5G3Z5AEQO2AVVBQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:51.765229559 (253h48m54.466822016s) [noise=true , address=145.79.227.26:4655 , piece_id=SYKVYBTUO2M3VZ34MHIRAMEGVGBM6OZWUYQ46C35VSZYETX2FEZQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:52.009059018 (253h48m54.710651469s) [noise=true , address=145.79.227.6:4258 , piece_id=ADOYO4XPHEUPCBE7RR6IM4WIQXCAFSAYC7OVY56PD62Y43HDWTOA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:52.009456880 (253h48m54.711049319s) [noise=true , address=145.79.227.28:4726 , piece_id=53QO5GLB3GC64YZ3Z6ALE3O4KSPDNHSDE7QKWMX7SPHYWXLCDBTA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:52.249419947 (253h48m54.9510124s) [noise=true , address=145.79.227.13:4803 , piece_id=ZPOECQBGXDFAM5MYW5CM73XRS56R44CG4UD74SCHBQEVTCGPEBHQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:52.747418960 (253h48m55.44901141s) [noise=true , address=145.79.227.20:4438 , piece_id=X26D2AKZ2CXYBKKBTFP6TB3MWL5LSW4JKGGZN43CIU3AU5IN3JWQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:53.507082237 (253h48m56.208674693s) [noise=true , address=145.79.227.30:4762 , piece_id=M2V6D3VWJMURCFRK5YFNVMAO2AIOBM2GWSL6OBZJE5HUY77IJLFQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:53.566714417 (253h48m56.268306873s) [noise=true , address=145.79.227.32:4835 , piece_id=SQ237HBZSYOFGIJHNRBJ3W4R5YVMQ6QLAMCJRSMLMQ4E76RFVFDA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:30:19.906744987 (253h49m22.608337469s) [noise=true , address=145.79.227.10:4403 , piece_id=DKPD6QFB4VDYUTF46OQZIYDLXA6XTSWDOMZDI2O35RIWIQSYWUOQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 139.59.144.22:35544->145.79.227.10:4403: use of closed network connection; write tcp 139.59.144.22:35544->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35544->145.79.227.10:4403: use of closed network connection Nov 10 01:30:19.906915176 (253h49m22.608507655s) [noise=true , address=145.79.227.8:4331 , piece_id=AHJM74RULWU2OLNPHSARMLJWUB5L343PESDONTTWECSQB7SI4Y2A, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: write tcp 139.59.144.22:57352->145.79.227.8:4331: use of closed network connection; write tcp 139.59.144.22:57352->145.79.227.8:4331: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57352->145.79.227.8:4331: use of closed network connection Nov 10 01:30:19.907148249 (253h49m22.608740766s) [noise=true , address=145.79.227.14:4837 , piece_id=6UJFDJPFALMG7HYK5DBZU7VV4GU27ROOFXOLG7WVSRQE6JBZMYKA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: read tcp 139.59.144.22:48446->145.79.227.14:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:30:19.907253647 (253h49m22.608846101s) [noise=true , address=145.79.227.16:4911 , piece_id=UIG36LK3PQNRTF5F47JI6UM5ITPLPPQNB3R4ORBHSLCLAST7NAPA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 139.59.144.22:38920->145.79.227.16:4911: use of closed network connection; write tcp 139.59.144.22:38920->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38920->145.79.227.16:4911: use of closed network connection Nov 10 01:30:19.907481275 (253h49m22.609073737s) [noise=true , address=145.79.227.21:4474 , piece_id=ITRP3SCJJ7JRC2SW54DV7CWXTVZHVJDGFSPA2VDR45CNX6UAAG3Q, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: context deadline exceeded; context deadline exceeded Nov 10 01:30:19.907515860 (253h49m22.609108309s) [seg_pos={0 0}, upload=223211]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 10 01:30:20.026141819 (253h49m22.727734268s) [upload=223211]: Done uploading segments... Nov 10 01:30:20.026177189 (253h49m22.727769638s) [upload=223211]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.330091965s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................................X.............XX..] duration: 2.459460998s trace: https://jaeger.dev.storj.io/trace/c055a066e80622b FAIL (2025-11-10 01:31:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 10 01:29:50.180949372 (253h48m52.882541824s) [upload=223211]: Starting upload Nov 10 01:29:50.181032473 (253h48m52.882624934s) [upload=223211]: Done starting upload Nov 10 01:29:50.181156682 (253h48m52.882749125s) [upload=223211]: Uploading segments... Nov 10 01:29:50.181465875 (253h48m52.883058315s) [upload=223211]: Got next segment. Inline: false Nov 10 01:29:50.181499404 (253h48m52.883091854s) [upload=223211]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 10 01:29:50.181504566 (253h48m52.883097006s) [upload=223211]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 10 01:29:50.288289364 (253h48m52.989881813s) [seg_pos={0 0}, upload=223211]: Begin upload segment... Nov 10 01:29:50.288363675 (253h48m52.989956127s) [seg_pos={0 0}, upload=223211]: Done begin upload segment. Nov 10 01:29:50.288400256 (253h48m52.989992693s) [noise=true , address=145.79.227.21:4474 , piece_id=ITRP3SCJJ7JRC2SW54DV7CWXTVZHVJDGFSPA2VDR45CNX6UAAG3Q, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.288416601 (253h48m52.990009056s) [noise=true , address=145.79.227.20:4438 , piece_id=X26D2AKZ2CXYBKKBTFP6TB3MWL5LSW4JKGGZN43CIU3AU5IN3JWQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.288921316 (253h48m52.990513768s) [noise=true , address=77.90.31.10:4520 , piece_id=FWC3GNZIQHRM3TBWUI42BYKLCNZ3LMYZNOMAFZEGEZS2LDWMHYBA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.289398939 (253h48m52.99099142s) [noise=true , address=145.79.227.13:4803 , piece_id=ZPOECQBGXDFAM5MYW5CM73XRS56R44CG4UD74SCHBQEVTCGPEBHQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.289886286 (253h48m52.991478737s) [noise=true , address=145.79.227.10:4403 , piece_id=DKPD6QFB4VDYUTF46OQZIYDLXA6XTSWDOMZDI2O35RIWIQSYWUOQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.290023453 (253h48m52.991615891s) [noise=true , address=145.79.227.26:4655 , piece_id=SYKVYBTUO2M3VZ34MHIRAMEGVGBM6OZWUYQ46C35VSZYETX2FEZQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.290402347 (253h48m52.991994805s) [noise=true , address=145.79.227.15:4876 , piece_id=THTJQZY5CJFDOZ64OI7RD3VP5EEOKHVT4T6MS5G3Z5AEQO2AVVBQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.290460076 (253h48m52.992052527s) [noise=true , address=77.90.31.21:4878 , piece_id=LRYUY6I45PIRGCLW6TH5DT4NZ4BCG7FSWTQ5KNQV6YDWQVVCRPCA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.290860429 (253h48m52.992452885s) [noise=true , address=145.79.227.14:4837 , piece_id=6UJFDJPFALMG7HYK5DBZU7VV4GU27ROOFXOLG7WVSRQE6JBZMYKA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.291265673 (253h48m52.992858124s) [noise=true , address=145.79.227.30:4762 , piece_id=M2V6D3VWJMURCFRK5YFNVMAO2AIOBM2GWSL6OBZJE5HUY77IJLFQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.291688246 (253h48m52.993280698s) [noise=true , address=145.79.227.28:4726 , piece_id=53QO5GLB3GC64YZ3Z6ALE3O4KSPDNHSDE7QKWMX7SPHYWXLCDBTA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.292140757 (253h48m52.993733216s) [noise=true , address=145.79.227.32:4835 , piece_id=SQ237HBZSYOFGIJHNRBJ3W4R5YVMQ6QLAMCJRSMLMQ4E76RFVFDA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.292895942 (253h48m52.994488396s) [noise=true , address=145.79.227.8:4331 , piece_id=AHJM74RULWU2OLNPHSARMLJWUB5L343PESDONTTWECSQB7SI4Y2A, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.293619050 (253h48m52.995211507s) [noise=true , address=145.79.227.16:4911 , piece_id=UIG36LK3PQNRTF5F47JI6UM5ITPLPPQNB3R4ORBHSLCLAST7NAPA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.294091314 (253h48m52.995683765s) [noise=true , address=77.90.31.15:4699 , piece_id=ZTT7EEHL2PBGXEEDZ5GOSHHCCQLSSIAMHK7AUDDFPCDS75FYQLOA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.294213128 (253h48m52.995805644s) [noise=true , address=145.79.227.6:4258 , piece_id=ADOYO4XPHEUPCBE7RR6IM4WIQXCAFSAYC7OVY56PD62Y43HDWTOA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.898598865 (253h48m53.600191317s) [upload=223211]: Next returned nil segment Nov 10 01:29:50.898616587 (253h48m53.600209045s) [upload=223211]: Waiting for error group managing segments... Nov 10 01:29:51.030451931 (253h48m53.732044379s) [noise=true , address=77.90.31.10:4520 , piece_id=FWC3GNZIQHRM3TBWUI42BYKLCNZ3LMYZNOMAFZEGEZS2LDWMHYBA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:51.051603630 (253h48m53.753196085s) [noise=true , address=77.90.31.15:4699 , piece_id=ZTT7EEHL2PBGXEEDZ5GOSHHCCQLSSIAMHK7AUDDFPCDS75FYQLOA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:51.051711369 (253h48m53.75330382s) [noise=true , address=77.90.31.21:4878 , piece_id=LRYUY6I45PIRGCLW6TH5DT4NZ4BCG7FSWTQ5KNQV6YDWQVVCRPCA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:51.532787951 (253h48m54.234380399s) [noise=true , address=145.79.227.15:4876 , piece_id=THTJQZY5CJFDOZ64OI7RD3VP5EEOKHVT4T6MS5G3Z5AEQO2AVVBQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:51.765229559 (253h48m54.466822016s) [noise=true , address=145.79.227.26:4655 , piece_id=SYKVYBTUO2M3VZ34MHIRAMEGVGBM6OZWUYQ46C35VSZYETX2FEZQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:52.009059018 (253h48m54.710651469s) [noise=true , address=145.79.227.6:4258 , piece_id=ADOYO4XPHEUPCBE7RR6IM4WIQXCAFSAYC7OVY56PD62Y43HDWTOA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:52.009456880 (253h48m54.711049319s) [noise=true , address=145.79.227.28:4726 , piece_id=53QO5GLB3GC64YZ3Z6ALE3O4KSPDNHSDE7QKWMX7SPHYWXLCDBTA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:52.249419947 (253h48m54.9510124s) [noise=true , address=145.79.227.13:4803 , piece_id=ZPOECQBGXDFAM5MYW5CM73XRS56R44CG4UD74SCHBQEVTCGPEBHQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:52.747418960 (253h48m55.44901141s) [noise=true , address=145.79.227.20:4438 , piece_id=X26D2AKZ2CXYBKKBTFP6TB3MWL5LSW4JKGGZN43CIU3AU5IN3JWQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:53.507082237 (253h48m56.208674693s) [noise=true , address=145.79.227.30:4762 , piece_id=M2V6D3VWJMURCFRK5YFNVMAO2AIOBM2GWSL6OBZJE5HUY77IJLFQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:53.566714417 (253h48m56.268306873s) [noise=true , address=145.79.227.32:4835 , piece_id=SQ237HBZSYOFGIJHNRBJ3W4R5YVMQ6QLAMCJRSMLMQ4E76RFVFDA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:30:19.906744987 (253h49m22.608337469s) [noise=true , address=145.79.227.10:4403 , piece_id=DKPD6QFB4VDYUTF46OQZIYDLXA6XTSWDOMZDI2O35RIWIQSYWUOQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 139.59.144.22:35544->145.79.227.10:4403: use of closed network connection; write tcp 139.59.144.22:35544->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35544->145.79.227.10:4403: use of closed network connection Nov 10 01:30:19.906915176 (253h49m22.608507655s) [noise=true , address=145.79.227.8:4331 , piece_id=AHJM74RULWU2OLNPHSARMLJWUB5L343PESDONTTWECSQB7SI4Y2A, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: write tcp 139.59.144.22:57352->145.79.227.8:4331: use of closed network connection; write tcp 139.59.144.22:57352->145.79.227.8:4331: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57352->145.79.227.8:4331: use of closed network connection Nov 10 01:30:19.907148249 (253h49m22.608740766s) [noise=true , address=145.79.227.14:4837 , piece_id=6UJFDJPFALMG7HYK5DBZU7VV4GU27ROOFXOLG7WVSRQE6JBZMYKA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: read tcp 139.59.144.22:48446->145.79.227.14:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:30:19.907253647 (253h49m22.608846101s) [noise=true , address=145.79.227.16:4911 , piece_id=UIG36LK3PQNRTF5F47JI6UM5ITPLPPQNB3R4ORBHSLCLAST7NAPA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 139.59.144.22:38920->145.79.227.16:4911: use of closed network connection; write tcp 139.59.144.22:38920->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38920->145.79.227.16:4911: use of closed network connection Nov 10 01:30:19.907481275 (253h49m22.609073737s) [noise=true , address=145.79.227.21:4474 , piece_id=ITRP3SCJJ7JRC2SW54DV7CWXTVZHVJDGFSPA2VDR45CNX6UAAG3Q, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: context deadline exceeded; context deadline exceeded Nov 10 01:30:19.907515860 (253h49m22.609108309s) [seg_pos={0 0}, upload=223211]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 10 01:30:20.026141819 (253h49m22.727734268s) [upload=223211]: Done uploading segments... Nov 10 01:30:20.026177189 (253h49m22.727769638s) [upload=223211]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.459487388s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................................X.............XX.] duration: 2.354822164s trace: https://jaeger.dev.storj.io/trace/c055a066e80622b FAIL (2025-11-10 01:30:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 10 01:29:50.180949372 (253h48m52.882541824s) [upload=223211]: Starting upload Nov 10 01:29:50.181032473 (253h48m52.882624934s) [upload=223211]: Done starting upload Nov 10 01:29:50.181156682 (253h48m52.882749125s) [upload=223211]: Uploading segments... Nov 10 01:29:50.181465875 (253h48m52.883058315s) [upload=223211]: Got next segment. Inline: false Nov 10 01:29:50.181499404 (253h48m52.883091854s) [upload=223211]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 10 01:29:50.181504566 (253h48m52.883097006s) [upload=223211]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 10 01:29:50.288289364 (253h48m52.989881813s) [seg_pos={0 0}, upload=223211]: Begin upload segment... Nov 10 01:29:50.288363675 (253h48m52.989956127s) [seg_pos={0 0}, upload=223211]: Done begin upload segment. Nov 10 01:29:50.288400256 (253h48m52.989992693s) [noise=true , address=145.79.227.21:4474 , piece_id=ITRP3SCJJ7JRC2SW54DV7CWXTVZHVJDGFSPA2VDR45CNX6UAAG3Q, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.288416601 (253h48m52.990009056s) [noise=true , address=145.79.227.20:4438 , piece_id=X26D2AKZ2CXYBKKBTFP6TB3MWL5LSW4JKGGZN43CIU3AU5IN3JWQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.288921316 (253h48m52.990513768s) [noise=true , address=77.90.31.10:4520 , piece_id=FWC3GNZIQHRM3TBWUI42BYKLCNZ3LMYZNOMAFZEGEZS2LDWMHYBA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.289398939 (253h48m52.99099142s) [noise=true , address=145.79.227.13:4803 , piece_id=ZPOECQBGXDFAM5MYW5CM73XRS56R44CG4UD74SCHBQEVTCGPEBHQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.289886286 (253h48m52.991478737s) [noise=true , address=145.79.227.10:4403 , piece_id=DKPD6QFB4VDYUTF46OQZIYDLXA6XTSWDOMZDI2O35RIWIQSYWUOQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.290023453 (253h48m52.991615891s) [noise=true , address=145.79.227.26:4655 , piece_id=SYKVYBTUO2M3VZ34MHIRAMEGVGBM6OZWUYQ46C35VSZYETX2FEZQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.290402347 (253h48m52.991994805s) [noise=true , address=145.79.227.15:4876 , piece_id=THTJQZY5CJFDOZ64OI7RD3VP5EEOKHVT4T6MS5G3Z5AEQO2AVVBQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.290460076 (253h48m52.992052527s) [noise=true , address=77.90.31.21:4878 , piece_id=LRYUY6I45PIRGCLW6TH5DT4NZ4BCG7FSWTQ5KNQV6YDWQVVCRPCA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.290860429 (253h48m52.992452885s) [noise=true , address=145.79.227.14:4837 , piece_id=6UJFDJPFALMG7HYK5DBZU7VV4GU27ROOFXOLG7WVSRQE6JBZMYKA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.291265673 (253h48m52.992858124s) [noise=true , address=145.79.227.30:4762 , piece_id=M2V6D3VWJMURCFRK5YFNVMAO2AIOBM2GWSL6OBZJE5HUY77IJLFQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.291688246 (253h48m52.993280698s) [noise=true , address=145.79.227.28:4726 , piece_id=53QO5GLB3GC64YZ3Z6ALE3O4KSPDNHSDE7QKWMX7SPHYWXLCDBTA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.292140757 (253h48m52.993733216s) [noise=true , address=145.79.227.32:4835 , piece_id=SQ237HBZSYOFGIJHNRBJ3W4R5YVMQ6QLAMCJRSMLMQ4E76RFVFDA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.292895942 (253h48m52.994488396s) [noise=true , address=145.79.227.8:4331 , piece_id=AHJM74RULWU2OLNPHSARMLJWUB5L343PESDONTTWECSQB7SI4Y2A, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.293619050 (253h48m52.995211507s) [noise=true , address=145.79.227.16:4911 , piece_id=UIG36LK3PQNRTF5F47JI6UM5ITPLPPQNB3R4ORBHSLCLAST7NAPA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.294091314 (253h48m52.995683765s) [noise=true , address=77.90.31.15:4699 , piece_id=ZTT7EEHL2PBGXEEDZ5GOSHHCCQLSSIAMHK7AUDDFPCDS75FYQLOA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.294213128 (253h48m52.995805644s) [noise=true , address=145.79.227.6:4258 , piece_id=ADOYO4XPHEUPCBE7RR6IM4WIQXCAFSAYC7OVY56PD62Y43HDWTOA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.898598865 (253h48m53.600191317s) [upload=223211]: Next returned nil segment Nov 10 01:29:50.898616587 (253h48m53.600209045s) [upload=223211]: Waiting for error group managing segments... Nov 10 01:29:51.030451931 (253h48m53.732044379s) [noise=true , address=77.90.31.10:4520 , piece_id=FWC3GNZIQHRM3TBWUI42BYKLCNZ3LMYZNOMAFZEGEZS2LDWMHYBA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:51.051603630 (253h48m53.753196085s) [noise=true , address=77.90.31.15:4699 , piece_id=ZTT7EEHL2PBGXEEDZ5GOSHHCCQLSSIAMHK7AUDDFPCDS75FYQLOA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:51.051711369 (253h48m53.75330382s) [noise=true , address=77.90.31.21:4878 , piece_id=LRYUY6I45PIRGCLW6TH5DT4NZ4BCG7FSWTQ5KNQV6YDWQVVCRPCA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:51.532787951 (253h48m54.234380399s) [noise=true , address=145.79.227.15:4876 , piece_id=THTJQZY5CJFDOZ64OI7RD3VP5EEOKHVT4T6MS5G3Z5AEQO2AVVBQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:51.765229559 (253h48m54.466822016s) [noise=true , address=145.79.227.26:4655 , piece_id=SYKVYBTUO2M3VZ34MHIRAMEGVGBM6OZWUYQ46C35VSZYETX2FEZQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:52.009059018 (253h48m54.710651469s) [noise=true , address=145.79.227.6:4258 , piece_id=ADOYO4XPHEUPCBE7RR6IM4WIQXCAFSAYC7OVY56PD62Y43HDWTOA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:52.009456880 (253h48m54.711049319s) [noise=true , address=145.79.227.28:4726 , piece_id=53QO5GLB3GC64YZ3Z6ALE3O4KSPDNHSDE7QKWMX7SPHYWXLCDBTA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:52.249419947 (253h48m54.9510124s) [noise=true , address=145.79.227.13:4803 , piece_id=ZPOECQBGXDFAM5MYW5CM73XRS56R44CG4UD74SCHBQEVTCGPEBHQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:52.747418960 (253h48m55.44901141s) [noise=true , address=145.79.227.20:4438 , piece_id=X26D2AKZ2CXYBKKBTFP6TB3MWL5LSW4JKGGZN43CIU3AU5IN3JWQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:53.507082237 (253h48m56.208674693s) [noise=true , address=145.79.227.30:4762 , piece_id=M2V6D3VWJMURCFRK5YFNVMAO2AIOBM2GWSL6OBZJE5HUY77IJLFQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:53.566714417 (253h48m56.268306873s) [noise=true , address=145.79.227.32:4835 , piece_id=SQ237HBZSYOFGIJHNRBJ3W4R5YVMQ6QLAMCJRSMLMQ4E76RFVFDA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:30:19.906744987 (253h49m22.608337469s) [noise=true , address=145.79.227.10:4403 , piece_id=DKPD6QFB4VDYUTF46OQZIYDLXA6XTSWDOMZDI2O35RIWIQSYWUOQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 139.59.144.22:35544->145.79.227.10:4403: use of closed network connection; write tcp 139.59.144.22:35544->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35544->145.79.227.10:4403: use of closed network connection Nov 10 01:30:19.906915176 (253h49m22.608507655s) [noise=true , address=145.79.227.8:4331 , piece_id=AHJM74RULWU2OLNPHSARMLJWUB5L343PESDONTTWECSQB7SI4Y2A, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: write tcp 139.59.144.22:57352->145.79.227.8:4331: use of closed network connection; write tcp 139.59.144.22:57352->145.79.227.8:4331: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57352->145.79.227.8:4331: use of closed network connection Nov 10 01:30:19.907148249 (253h49m22.608740766s) [noise=true , address=145.79.227.14:4837 , piece_id=6UJFDJPFALMG7HYK5DBZU7VV4GU27ROOFXOLG7WVSRQE6JBZMYKA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: read tcp 139.59.144.22:48446->145.79.227.14:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:30:19.907253647 (253h49m22.608846101s) [noise=true , address=145.79.227.16:4911 , piece_id=UIG36LK3PQNRTF5F47JI6UM5ITPLPPQNB3R4ORBHSLCLAST7NAPA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 139.59.144.22:38920->145.79.227.16:4911: use of closed network connection; write tcp 139.59.144.22:38920->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38920->145.79.227.16:4911: use of closed network connection Nov 10 01:30:19.907481275 (253h49m22.609073737s) [noise=true , address=145.79.227.21:4474 , piece_id=ITRP3SCJJ7JRC2SW54DV7CWXTVZHVJDGFSPA2VDR45CNX6UAAG3Q, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: context deadline exceeded; context deadline exceeded Nov 10 01:30:19.907515860 (253h49m22.609108309s) [seg_pos={0 0}, upload=223211]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 10 01:30:20.026141819 (253h49m22.727734268s) [upload=223211]: Done uploading segments... Nov 10 01:30:20.026177189 (253h49m22.727769638s) [upload=223211]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.354865619s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................................X.............XX] duration: 30.121254182s trace: https://jaeger.dev.storj.io/trace/c055a066e80622b FAIL (2025-11-10 01:30:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 10 01:29:50.180949372 (253h48m52.882541824s) [upload=223211]: Starting upload Nov 10 01:29:50.181032473 (253h48m52.882624934s) [upload=223211]: Done starting upload Nov 10 01:29:50.181156682 (253h48m52.882749125s) [upload=223211]: Uploading segments... Nov 10 01:29:50.181465875 (253h48m52.883058315s) [upload=223211]: Got next segment. Inline: false Nov 10 01:29:50.181499404 (253h48m52.883091854s) [upload=223211]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 10 01:29:50.181504566 (253h48m52.883097006s) [upload=223211]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 10 01:29:50.288289364 (253h48m52.989881813s) [seg_pos={0 0}, upload=223211]: Begin upload segment... Nov 10 01:29:50.288363675 (253h48m52.989956127s) [seg_pos={0 0}, upload=223211]: Done begin upload segment. Nov 10 01:29:50.288400256 (253h48m52.989992693s) [noise=true , address=145.79.227.21:4474 , piece_id=ITRP3SCJJ7JRC2SW54DV7CWXTVZHVJDGFSPA2VDR45CNX6UAAG3Q, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.288416601 (253h48m52.990009056s) [noise=true , address=145.79.227.20:4438 , piece_id=X26D2AKZ2CXYBKKBTFP6TB3MWL5LSW4JKGGZN43CIU3AU5IN3JWQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.288921316 (253h48m52.990513768s) [noise=true , address=77.90.31.10:4520 , piece_id=FWC3GNZIQHRM3TBWUI42BYKLCNZ3LMYZNOMAFZEGEZS2LDWMHYBA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.289398939 (253h48m52.99099142s) [noise=true , address=145.79.227.13:4803 , piece_id=ZPOECQBGXDFAM5MYW5CM73XRS56R44CG4UD74SCHBQEVTCGPEBHQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.289886286 (253h48m52.991478737s) [noise=true , address=145.79.227.10:4403 , piece_id=DKPD6QFB4VDYUTF46OQZIYDLXA6XTSWDOMZDI2O35RIWIQSYWUOQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.290023453 (253h48m52.991615891s) [noise=true , address=145.79.227.26:4655 , piece_id=SYKVYBTUO2M3VZ34MHIRAMEGVGBM6OZWUYQ46C35VSZYETX2FEZQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.290402347 (253h48m52.991994805s) [noise=true , address=145.79.227.15:4876 , piece_id=THTJQZY5CJFDOZ64OI7RD3VP5EEOKHVT4T6MS5G3Z5AEQO2AVVBQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.290460076 (253h48m52.992052527s) [noise=true , address=77.90.31.21:4878 , piece_id=LRYUY6I45PIRGCLW6TH5DT4NZ4BCG7FSWTQ5KNQV6YDWQVVCRPCA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.290860429 (253h48m52.992452885s) [noise=true , address=145.79.227.14:4837 , piece_id=6UJFDJPFALMG7HYK5DBZU7VV4GU27ROOFXOLG7WVSRQE6JBZMYKA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.291265673 (253h48m52.992858124s) [noise=true , address=145.79.227.30:4762 , piece_id=M2V6D3VWJMURCFRK5YFNVMAO2AIOBM2GWSL6OBZJE5HUY77IJLFQ, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.291688246 (253h48m52.993280698s) [noise=true , address=145.79.227.28:4726 , piece_id=53QO5GLB3GC64YZ3Z6ALE3O4KSPDNHSDE7QKWMX7SPHYWXLCDBTA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.292140757 (253h48m52.993733216s) [noise=true , address=145.79.227.32:4835 , piece_id=SQ237HBZSYOFGIJHNRBJ3W4R5YVMQ6QLAMCJRSMLMQ4E76RFVFDA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.292895942 (253h48m52.994488396s) [noise=true , address=145.79.227.8:4331 , piece_id=AHJM74RULWU2OLNPHSARMLJWUB5L343PESDONTTWECSQB7SI4Y2A, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.293619050 (253h48m52.995211507s) [noise=true , address=145.79.227.16:4911 , piece_id=UIG36LK3PQNRTF5F47JI6UM5ITPLPPQNB3R4ORBHSLCLAST7NAPA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.294091314 (253h48m52.995683765s) [noise=true , address=77.90.31.15:4699 , piece_id=ZTT7EEHL2PBGXEEDZ5GOSHHCCQLSSIAMHK7AUDDFPCDS75FYQLOA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.294213128 (253h48m52.995805644s) [noise=true , address=145.79.227.6:4258 , piece_id=ADOYO4XPHEUPCBE7RR6IM4WIQXCAFSAYC7OVY56PD62Y43HDWTOA, seg_pos={0 0}, upload=223211]: Uploading piece... Nov 10 01:29:50.898598865 (253h48m53.600191317s) [upload=223211]: Next returned nil segment Nov 10 01:29:50.898616587 (253h48m53.600209045s) [upload=223211]: Waiting for error group managing segments... Nov 10 01:29:51.030451931 (253h48m53.732044379s) [noise=true , address=77.90.31.10:4520 , piece_id=FWC3GNZIQHRM3TBWUI42BYKLCNZ3LMYZNOMAFZEGEZS2LDWMHYBA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:51.051603630 (253h48m53.753196085s) [noise=true , address=77.90.31.15:4699 , piece_id=ZTT7EEHL2PBGXEEDZ5GOSHHCCQLSSIAMHK7AUDDFPCDS75FYQLOA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:51.051711369 (253h48m53.75330382s) [noise=true , address=77.90.31.21:4878 , piece_id=LRYUY6I45PIRGCLW6TH5DT4NZ4BCG7FSWTQ5KNQV6YDWQVVCRPCA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:51.532787951 (253h48m54.234380399s) [noise=true , address=145.79.227.15:4876 , piece_id=THTJQZY5CJFDOZ64OI7RD3VP5EEOKHVT4T6MS5G3Z5AEQO2AVVBQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:51.765229559 (253h48m54.466822016s) [noise=true , address=145.79.227.26:4655 , piece_id=SYKVYBTUO2M3VZ34MHIRAMEGVGBM6OZWUYQ46C35VSZYETX2FEZQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:52.009059018 (253h48m54.710651469s) [noise=true , address=145.79.227.6:4258 , piece_id=ADOYO4XPHEUPCBE7RR6IM4WIQXCAFSAYC7OVY56PD62Y43HDWTOA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:52.009456880 (253h48m54.711049319s) [noise=true , address=145.79.227.28:4726 , piece_id=53QO5GLB3GC64YZ3Z6ALE3O4KSPDNHSDE7QKWMX7SPHYWXLCDBTA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:52.249419947 (253h48m54.9510124s) [noise=true , address=145.79.227.13:4803 , piece_id=ZPOECQBGXDFAM5MYW5CM73XRS56R44CG4UD74SCHBQEVTCGPEBHQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:52.747418960 (253h48m55.44901141s) [noise=true , address=145.79.227.20:4438 , piece_id=X26D2AKZ2CXYBKKBTFP6TB3MWL5LSW4JKGGZN43CIU3AU5IN3JWQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:53.507082237 (253h48m56.208674693s) [noise=true , address=145.79.227.30:4762 , piece_id=M2V6D3VWJMURCFRK5YFNVMAO2AIOBM2GWSL6OBZJE5HUY77IJLFQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:29:53.566714417 (253h48m56.268306873s) [noise=true , address=145.79.227.32:4835 , piece_id=SQ237HBZSYOFGIJHNRBJ3W4R5YVMQ6QLAMCJRSMLMQ4E76RFVFDA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: <nil> Nov 10 01:30:19.906744987 (253h49m22.608337469s) [noise=true , address=145.79.227.10:4403 , piece_id=DKPD6QFB4VDYUTF46OQZIYDLXA6XTSWDOMZDI2O35RIWIQSYWUOQ, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 139.59.144.22:35544->145.79.227.10:4403: use of closed network connection; write tcp 139.59.144.22:35544->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35544->145.79.227.10:4403: use of closed network connection Nov 10 01:30:19.906915176 (253h49m22.608507655s) [noise=true , address=145.79.227.8:4331 , piece_id=AHJM74RULWU2OLNPHSARMLJWUB5L343PESDONTTWECSQB7SI4Y2A, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: write tcp 139.59.144.22:57352->145.79.227.8:4331: use of closed network connection; write tcp 139.59.144.22:57352->145.79.227.8:4331: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57352->145.79.227.8:4331: use of closed network connection Nov 10 01:30:19.907148249 (253h49m22.608740766s) [noise=true , address=145.79.227.14:4837 , piece_id=6UJFDJPFALMG7HYK5DBZU7VV4GU27ROOFXOLG7WVSRQE6JBZMYKA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: read tcp 139.59.144.22:48446->145.79.227.14:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:30:19.907253647 (253h49m22.608846101s) [noise=true , address=145.79.227.16:4911 , piece_id=UIG36LK3PQNRTF5F47JI6UM5ITPLPPQNB3R4ORBHSLCLAST7NAPA, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 139.59.144.22:38920->145.79.227.16:4911: use of closed network connection; write tcp 139.59.144.22:38920->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38920->145.79.227.16:4911: use of closed network connection Nov 10 01:30:19.907481275 (253h49m22.609073737s) [noise=true , address=145.79.227.21:4474 , piece_id=ITRP3SCJJ7JRC2SW54DV7CWXTVZHVJDGFSPA2VDR45CNX6UAAG3Q, seg_pos={0 0}, upload=223211]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: context deadline exceeded; context deadline exceeded Nov 10 01:30:19.907515860 (253h49m22.609108309s) [seg_pos={0 0}, upload=223211]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 10 01:30:20.026141819 (253h49m22.727734268s) [upload=223211]: Done uploading segments... Nov 10 01:30:20.026177189 (253h49m22.727769638s) [upload=223211]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.121285844s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X.............X] duration: 30.11728564s trace: https://jaeger.dev.storj.io/trace/6d03ef76b9a663a9 FAIL (2025-11-10 01:29:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== ]: Uploading segments... Nov 10 01:29:17.390363081 (253h48m20.091955536s) [upload=223204]: Got next segment. Inline: false Nov 10 01:29:17.390448920 (253h48m20.092041375s) [upload=223204]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 10 01:29:17.390457287 (253h48m20.092049747s) [upload=223204]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 10 01:29:17.499523206 (253h48m20.201115685s) [seg_pos={0 0}, upload=223204]: Begin upload segment... Nov 10 01:29:17.499582912 (253h48m20.201175383s) [seg_pos={0 0}, upload=223204]: Done begin upload segment. Nov 10 01:29:17.499620966 (253h48m20.201213421s) [noise=true , address=77.90.31.19:4843 , piece_id=EXXCTYS2S5PAQDNNV4JBCYBU53VLPONFLN7AVSWAYVSPEUOJA43A, seg_pos={0 0}, upload=223204]: Uploading piece... Nov 10 01:29:17.499695018 (253h48m20.201287472s) [noise=true , address=145.79.227.31:4799 , piece_id=BOEQDJCDVG2MCKNYAU6T2YHIOP42ELWHGA5BC2HUCVJPDPMYK4FQ, seg_pos={0 0}, upload=223204]: Uploading piece... Nov 10 01:29:17.500200855 (253h48m20.201793313s) [noise=true , address=145.79.227.28:4727 , piece_id=KBMQADPXHY2HKHNGDNUOP45U5QCVDW6CKJQYCZFRPUYP6TBI5YFA, seg_pos={0 0}, upload=223204]: Uploading piece... Nov 10 01:29:17.500706729 (253h48m20.202299188s) [noise=true , address=145.79.227.32:4834 , piece_id=BUYANV3X6ZIBLF5RV67IJGNGNOLINUTN3255GNZDU3AVACEFEH2A, seg_pos={0 0}, upload=223204]: Uploading piece... Nov 10 01:29:17.501003899 (253h48m20.202596351s) [noise=true , address=45.91.116.4:4939 , piece_id=SAQLYGDVB7MJ72E5YP33UHMX4OWMHDA3RTIEOIPISUHKRDNVVSPA, seg_pos={0 0}, upload=223204]: Uploading piece... Nov 10 01:29:17.501142092 (253h48m20.202734551s) [noise=true , address=77.90.31.21:4877 , piece_id=S3GI7G6W46G2IYE2MDP6GLNMUJYALHIZCBQXCKKL26INE6HCBBVA, seg_pos={0 0}, upload=223204]: Uploading piece... Nov 10 01:29:17.501488744 (253h48m20.203081202s) [noise=true , address=145.79.227.10:4401 , piece_id=JUJ3KLGQMV7YN5POUFWSH7KSFT6ENYHI6YHNRIX6AYKZ3DITR5AQ, seg_pos={0 0}, upload=223204]: Uploading piece... Nov 10 01:29:17.501579261 (253h48m20.203171719s) [noise=true , address=145.79.227.23:4545 , piece_id=6III64TB3TDNFZOHY3YFCP7KXC2KBFEUS6EUJTLCLCWAT66DPTMA, seg_pos={0 0}, upload=223204]: Uploading piece... Nov 10 01:29:17.502018130 (253h48m20.203610571s) [noise=true , address=145.79.227.21:4473 , piece_id=OTBV57J7XLX5RCPHEWHOF3MMQMLGTFQ337TTVWK75EX7AAZ2VJWA, seg_pos={0 0}, upload=223204]: Uploading piece... Nov 10 01:29:17.502114122 (253h48m20.20370658s) [noise=true , address=145.79.227.16:4911 , piece_id=RDRS3NWZCNRD3M4DAYYIFW6DPK2CJJGABRIUKB5ZT5LZTKGOETHA, seg_pos={0 0}, upload=223204]: Uploading piece... Nov 10 01:29:17.502487281 (253h48m20.204079732s) [noise=true , address=145.79.227.28:4726 , piece_id=WDZ3XP5A6KB6JGBZD7RKWGH5WEQK6SBSIOHFBOMQMBTKJ2HAFRVA, seg_pos={0 0}, upload=223204]: Uploading piece... Nov 10 01:29:17.502567887 (253h48m20.204160345s) [noise=true , address=145.79.227.27:4691 , piece_id=25452OOFXJQINKR52IKFYCFSAJGF2ZLRHXJHNHMU6CT73QRNP4HA, seg_pos={0 0}, upload=223204]: Uploading piece... Nov 10 01:29:17.502845305 (253h48m20.204437756s) [noise=true , address=145.79.227.30:4761 , piece_id=TATAVTFW6K2CXAODLI7RPEYQDX7RHPLWSFHF26OEC74LJLZKDG6Q, seg_pos={0 0}, upload=223204]: Uploading piece... Nov 10 01:29:17.503166586 (253h48m20.20475902s) [noise=true , address=145.79.227.6:4258 , piece_id=JGF6CXRGK46JSCKCSBCPDNLRJ2CTZ6MHDHW25BQ2AE6E3GGIPV2Q, seg_pos={0 0}, upload=223204]: Uploading piece... Nov 10 01:29:17.503223369 (253h48m20.204815821s) [noise=true , address=45.91.116.9:4263 , piece_id=LVTUONQDKSPTSMBM7VBIZ2CMNYFOTKVP2SEQAG5HQDDGWO3PFS3A, seg_pos={0 0}, upload=223204]: Uploading piece... Nov 10 01:29:17.503634394 (253h48m20.205226845s) [noise=true , address=77.90.31.11:4555 , piece_id=VFW4KBRZRCKPKPDPDHZLG3ZYHYETVSZAR33BSXZB5DJ47EW4A2VA, seg_pos={0 0}, upload=223204]: Uploading piece... Nov 10 01:29:18.125953878 (253h48m20.827546331s) [upload=223204]: Next returned nil segment Nov 10 01:29:18.125972556 (253h48m20.82756501s) [upload=223204]: Waiting for error group managing segments... Nov 10 01:29:18.246980660 (253h48m20.948573108s) [noise=true , address=77.90.31.19:4843 , piece_id=EXXCTYS2S5PAQDNNV4JBCYBU53VLPONFLN7AVSWAYVSPEUOJA43A, seg_pos={0 0}, upload=223204]: Done uploading piece. err: <nil> Nov 10 01:29:18.254132715 (253h48m20.955725165s) [noise=true , address=77.90.31.21:4877 , piece_id=S3GI7G6W46G2IYE2MDP6GLNMUJYALHIZCBQXCKKL26INE6HCBBVA, seg_pos={0 0}, upload=223204]: Done uploading piece. err: <nil> Nov 10 01:29:18.254227788 (253h48m20.955820238s) [noise=true , address=45.91.116.9:4263 , piece_id=LVTUONQDKSPTSMBM7VBIZ2CMNYFOTKVP2SEQAG5HQDDGWO3PFS3A, seg_pos={0 0}, upload=223204]: Done uploading piece. err: <nil> Nov 10 01:29:18.257641712 (253h48m20.95923416s) [noise=true , address=45.91.116.4:4939 , piece_id=SAQLYGDVB7MJ72E5YP33UHMX4OWMHDA3RTIEOIPISUHKRDNVVSPA, seg_pos={0 0}, upload=223204]: Done uploading piece. err: <nil> Nov 10 01:29:18.261363697 (253h48m20.962956146s) [noise=true , address=77.90.31.11:4555 , piece_id=VFW4KBRZRCKPKPDPDHZLG3ZYHYETVSZAR33BSXZB5DJ47EW4A2VA, seg_pos={0 0}, upload=223204]: Done uploading piece. err: <nil> Nov 10 01:29:18.450470419 (253h48m21.152062884s) [noise=true , address=145.79.227.10:4401 , piece_id=JUJ3KLGQMV7YN5POUFWSH7KSFT6ENYHI6YHNRIX6AYKZ3DITR5AQ, seg_pos={0 0}, upload=223204]: Done uploading piece. err: <nil> Nov 10 01:29:18.471947108 (253h48m21.173539557s) [noise=true , address=145.79.227.23:4545 , piece_id=6III64TB3TDNFZOHY3YFCP7KXC2KBFEUS6EUJTLCLCWAT66DPTMA, seg_pos={0 0}, upload=223204]: Done uploading piece. err: <nil> Nov 10 01:29:18.556617793 (253h48m21.258210279s) [noise=true , address=145.79.227.27:4691 , piece_id=25452OOFXJQINKR52IKFYCFSAJGF2ZLRHXJHNHMU6CT73QRNP4HA, seg_pos={0 0}, upload=223204]: Done uploading piece. err: <nil> Nov 10 01:29:18.564488413 (253h48m21.266080862s) [noise=true , address=145.79.227.16:4911 , piece_id=RDRS3NWZCNRD3M4DAYYIFW6DPK2CJJGABRIUKB5ZT5LZTKGOETHA, seg_pos={0 0}, upload=223204]: Done uploading piece. err: <nil> Nov 10 01:29:18.761881777 (253h48m21.463474231s) [noise=true , address=145.79.227.28:4726 , piece_id=WDZ3XP5A6KB6JGBZD7RKWGH5WEQK6SBSIOHFBOMQMBTKJ2HAFRVA, seg_pos={0 0}, upload=223204]: Done uploading piece. err: <nil> Nov 10 01:29:25.972559617 (253h48m28.674152066s) [noise=true , address=145.79.227.28:4727 , piece_id=KBMQADPXHY2HKHNGDNUOP45U5QCVDW6CKJQYCZFRPUYP6TBI5YFA, seg_pos={0 0}, upload=223204]: Done uploading piece. err: <nil> Nov 10 01:29:47.100154030 (253h48m49.801746512s) [noise=true , address=145.79.227.31:4799 , piece_id=BOEQDJCDVG2MCKNYAU6T2YHIOP42ELWHGA5BC2HUCVJPDPMYK4FQ, seg_pos={0 0}, upload=223204]: Done uploading piece. err: ecclient: upload failed (node:1vMQm9RufCEcHca9k1JEhyED9MfYjvZcT2MFYic3PmEKyUoC7X, address:145.79.227.31:4799): protocol: write tcp 139.59.144.22:50364->145.79.227.31:4799: use of closed network connection; write tcp 139.59.144.22:50364->145.79.227.31:4799: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50364->145.79.227.31:4799: use of closed network connection Nov 10 01:29:47.100426041 (253h48m49.8020185s) [noise=true , address=145.79.227.21:4473 , piece_id=OTBV57J7XLX5RCPHEWHOF3MMQMLGTFQ337TTVWK75EX7AAZ2VJWA, seg_pos={0 0}, upload=223204]: Done uploading piece. err: ecclient: upload failed (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk, address:145.79.227.21:4473): protocol: write tcp 139.59.144.22:34036->145.79.227.21:4473: use of closed network connection; write tcp 139.59.144.22:34036->145.79.227.21:4473: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34036->145.79.227.21:4473: use of closed network connection Nov 10 01:29:47.100607464 (253h48m49.802199924s) [noise=true , address=145.79.227.30:4761 , piece_id=TATAVTFW6K2CXAODLI7RPEYQDX7RHPLWSFHF26OEC74LJLZKDG6Q, seg_pos={0 0}, upload=223204]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 139.59.144.22:57244->145.79.227.30:4761: use of closed network connection; write tcp 139.59.144.22:57244->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57244->145.79.227.30:4761: use of closed network connection Nov 10 01:29:47.100747560 (253h48m49.802340012s) [noise=true , address=145.79.227.32:4834 , piece_id=BUYANV3X6ZIBLF5RV67IJGNGNOLINUTN3255GNZDU3AVACEFEH2A, seg_pos={0 0}, upload=223204]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: write tcp 139.59.144.22:37748->145.79.227.32:4834: use of closed network connection; write tcp 139.59.144.22:37748->145.79.227.32:4834: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37748->145.79.227.32:4834: use of closed network connection Nov 10 01:29:47.100766639 (253h48m49.802359077s) [noise=true , address=145.79.227.6:4258 , piece_id=JGF6CXRGK46JSCKCSBCPDNLRJ2CTZ6MHDHW25BQ2AE6E3GGIPV2Q, seg_pos={0 0}, upload=223204]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 139.59.144.22:48600->145.79.227.6:4258: use of closed network connection; write tcp 139.59.144.22:48600->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48600->145.79.227.6:4258: use of closed network connection Nov 10 01:29:47.100804627 (253h48m49.802397061s) [seg_pos={0 0}, upload=223204]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 10 01:29:47.214830903 (253h48m49.916423359s) [upload=223204]: Done uploading segments... Nov 10 01:29:47.214873163 (253h48m49.916465636s) [upload=223204]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.117312146s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.453945971s trace: https://jaeger.dev.storj.io/trace/3e23ce93b848e884 FAIL (2025-11-10 01:28:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 10 01:17:33.791953568 (253h36m36.493546021s) [upload=223030]: Starting upload Nov 10 01:17:33.792024560 (253h36m36.493617011s) [upload=223030]: Done starting upload Nov 10 01:17:33.792088057 (253h36m36.4936805s) [upload=223030]: Uploading segments... Nov 10 01:17:33.792513920 (253h36m36.494106378s) [upload=223030]: Got next segment. Inline: false Nov 10 01:17:33.792607652 (253h36m36.494200103s) [upload=223030]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 10 01:17:33.792615385 (253h36m36.494207835s) [upload=223030]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 10 01:17:33.904968320 (253h36m36.606560773s) [seg_pos={0 0}, upload=223030]: Begin upload segment... Nov 10 01:17:33.905008391 (253h36m36.606600841s) [seg_pos={0 0}, upload=223030]: Done begin upload segment. Nov 10 01:17:33.905031334 (253h36m36.606623791s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.905552396 (253h36m36.607144831s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.908076814 (253h36m36.609669269s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.909647138 (253h36m36.611239619s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910297142 (253h36m36.611889597s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910862225 (253h36m36.612454684s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911330437 (253h36m36.61292289s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911797731 (253h36m36.613390187s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.920749518 (253h36m36.622341975s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.930460228 (253h36m36.632052689s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.935906064 (253h36m36.637498526s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.952888411 (253h36m36.654501222s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958284571 (253h36m36.659877059s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958405123 (253h36m36.659997559s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960415899 (253h36m36.662008341s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960872922 (253h36m36.662465364s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:34.533147917 (253h36m37.234740374s) [upload=223030]: Next returned nil segment Nov 10 01:17:34.533604854 (253h36m37.235197304s) [upload=223030]: Waiting for error group managing segments... Nov 10 01:17:34.675100228 (253h36m37.376692692s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.677791419 (253h36m37.379383873s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.917050191 (253h36m37.61864264s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.929769222 (253h36m37.631361678s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.934998499 (253h36m37.636590952s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.039420506 (253h36m37.74101296s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.051494667 (253h36m37.753087116s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.054572229 (253h36m37.756164678s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.080485645 (253h36m37.782078106s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.120581463 (253h36m37.822173915s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.212846836 (253h36m37.914439285s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:18:03.505314517 (253h37m6.206906974s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection Nov 10 01:18:03.505444022 (253h37m6.207036475s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection Nov 10 01:18:03.505769555 (253h37m6.207362014s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.505907258 (253h37m6.207499716s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection Nov 10 01:18:03.506561789 (253h37m6.208154243s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: read tcp 139.59.144.22:39162->145.79.227.16:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.506606954 (253h37m6.208199394s) [seg_pos={0 0}, upload=223030]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 10 01:18:03.619588197 (253h37m6.321180649s) [upload=223030]: Done uploading segments... Nov 10 01:18:03.619624333 (253h37m6.321216788s) [upload=223030]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.453999908s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 17.745618124s trace: https://jaeger.dev.storj.io/trace/3e23ce93b848e884 FAIL (2025-11-10 01:27:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 10 01:17:33.791953568 (253h36m36.493546021s) [upload=223030]: Starting upload Nov 10 01:17:33.792024560 (253h36m36.493617011s) [upload=223030]: Done starting upload Nov 10 01:17:33.792088057 (253h36m36.4936805s) [upload=223030]: Uploading segments... Nov 10 01:17:33.792513920 (253h36m36.494106378s) [upload=223030]: Got next segment. Inline: false Nov 10 01:17:33.792607652 (253h36m36.494200103s) [upload=223030]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 10 01:17:33.792615385 (253h36m36.494207835s) [upload=223030]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 10 01:17:33.904968320 (253h36m36.606560773s) [seg_pos={0 0}, upload=223030]: Begin upload segment... Nov 10 01:17:33.905008391 (253h36m36.606600841s) [seg_pos={0 0}, upload=223030]: Done begin upload segment. Nov 10 01:17:33.905031334 (253h36m36.606623791s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.905552396 (253h36m36.607144831s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.908076814 (253h36m36.609669269s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.909647138 (253h36m36.611239619s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910297142 (253h36m36.611889597s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910862225 (253h36m36.612454684s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911330437 (253h36m36.61292289s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911797731 (253h36m36.613390187s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.920749518 (253h36m36.622341975s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.930460228 (253h36m36.632052689s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.935906064 (253h36m36.637498526s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.952888411 (253h36m36.654501222s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958284571 (253h36m36.659877059s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958405123 (253h36m36.659997559s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960415899 (253h36m36.662008341s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960872922 (253h36m36.662465364s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:34.533147917 (253h36m37.234740374s) [upload=223030]: Next returned nil segment Nov 10 01:17:34.533604854 (253h36m37.235197304s) [upload=223030]: Waiting for error group managing segments... Nov 10 01:17:34.675100228 (253h36m37.376692692s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.677791419 (253h36m37.379383873s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.917050191 (253h36m37.61864264s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.929769222 (253h36m37.631361678s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.934998499 (253h36m37.636590952s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.039420506 (253h36m37.74101296s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.051494667 (253h36m37.753087116s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.054572229 (253h36m37.756164678s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.080485645 (253h36m37.782078106s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.120581463 (253h36m37.822173915s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.212846836 (253h36m37.914439285s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:18:03.505314517 (253h37m6.206906974s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection Nov 10 01:18:03.505444022 (253h37m6.207036475s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection Nov 10 01:18:03.505769555 (253h37m6.207362014s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.505907258 (253h37m6.207499716s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection Nov 10 01:18:03.506561789 (253h37m6.208154243s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: read tcp 139.59.144.22:39162->145.79.227.16:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.506606954 (253h37m6.208199394s) [seg_pos={0 0}, upload=223030]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 10 01:18:03.619588197 (253h37m6.321180649s) [upload=223030]: Done uploading segments... Nov 10 01:18:03.619624333 (253h37m6.321216788s) [upload=223030]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 17.74565652s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.43734084s trace: https://jaeger.dev.storj.io/trace/3e23ce93b848e884 FAIL (2025-11-10 01:26:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 10 01:17:33.791953568 (253h36m36.493546021s) [upload=223030]: Starting upload Nov 10 01:17:33.792024560 (253h36m36.493617011s) [upload=223030]: Done starting upload Nov 10 01:17:33.792088057 (253h36m36.4936805s) [upload=223030]: Uploading segments... Nov 10 01:17:33.792513920 (253h36m36.494106378s) [upload=223030]: Got next segment. Inline: false Nov 10 01:17:33.792607652 (253h36m36.494200103s) [upload=223030]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 10 01:17:33.792615385 (253h36m36.494207835s) [upload=223030]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 10 01:17:33.904968320 (253h36m36.606560773s) [seg_pos={0 0}, upload=223030]: Begin upload segment... Nov 10 01:17:33.905008391 (253h36m36.606600841s) [seg_pos={0 0}, upload=223030]: Done begin upload segment. Nov 10 01:17:33.905031334 (253h36m36.606623791s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.905552396 (253h36m36.607144831s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.908076814 (253h36m36.609669269s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.909647138 (253h36m36.611239619s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910297142 (253h36m36.611889597s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910862225 (253h36m36.612454684s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911330437 (253h36m36.61292289s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911797731 (253h36m36.613390187s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.920749518 (253h36m36.622341975s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.930460228 (253h36m36.632052689s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.935906064 (253h36m36.637498526s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.952888411 (253h36m36.654501222s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958284571 (253h36m36.659877059s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958405123 (253h36m36.659997559s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960415899 (253h36m36.662008341s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960872922 (253h36m36.662465364s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:34.533147917 (253h36m37.234740374s) [upload=223030]: Next returned nil segment Nov 10 01:17:34.533604854 (253h36m37.235197304s) [upload=223030]: Waiting for error group managing segments... Nov 10 01:17:34.675100228 (253h36m37.376692692s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.677791419 (253h36m37.379383873s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.917050191 (253h36m37.61864264s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.929769222 (253h36m37.631361678s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.934998499 (253h36m37.636590952s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.039420506 (253h36m37.74101296s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.051494667 (253h36m37.753087116s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.054572229 (253h36m37.756164678s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.080485645 (253h36m37.782078106s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.120581463 (253h36m37.822173915s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.212846836 (253h36m37.914439285s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:18:03.505314517 (253h37m6.206906974s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection Nov 10 01:18:03.505444022 (253h37m6.207036475s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection Nov 10 01:18:03.505769555 (253h37m6.207362014s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.505907258 (253h37m6.207499716s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection Nov 10 01:18:03.506561789 (253h37m6.208154243s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: read tcp 139.59.144.22:39162->145.79.227.16:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.506606954 (253h37m6.208199394s) [seg_pos={0 0}, upload=223030]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 10 01:18:03.619588197 (253h37m6.321180649s) [upload=223030]: Done uploading segments... Nov 10 01:18:03.619624333 (253h37m6.321216788s) [upload=223030]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.437371964s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.515091979s trace: https://jaeger.dev.storj.io/trace/3e23ce93b848e884 FAIL (2025-11-10 01:26:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 10 01:17:33.791953568 (253h36m36.493546021s) [upload=223030]: Starting upload Nov 10 01:17:33.792024560 (253h36m36.493617011s) [upload=223030]: Done starting upload Nov 10 01:17:33.792088057 (253h36m36.4936805s) [upload=223030]: Uploading segments... Nov 10 01:17:33.792513920 (253h36m36.494106378s) [upload=223030]: Got next segment. Inline: false Nov 10 01:17:33.792607652 (253h36m36.494200103s) [upload=223030]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 10 01:17:33.792615385 (253h36m36.494207835s) [upload=223030]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 10 01:17:33.904968320 (253h36m36.606560773s) [seg_pos={0 0}, upload=223030]: Begin upload segment... Nov 10 01:17:33.905008391 (253h36m36.606600841s) [seg_pos={0 0}, upload=223030]: Done begin upload segment. Nov 10 01:17:33.905031334 (253h36m36.606623791s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.905552396 (253h36m36.607144831s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.908076814 (253h36m36.609669269s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.909647138 (253h36m36.611239619s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910297142 (253h36m36.611889597s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910862225 (253h36m36.612454684s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911330437 (253h36m36.61292289s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911797731 (253h36m36.613390187s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.920749518 (253h36m36.622341975s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.930460228 (253h36m36.632052689s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.935906064 (253h36m36.637498526s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.952888411 (253h36m36.654501222s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958284571 (253h36m36.659877059s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958405123 (253h36m36.659997559s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960415899 (253h36m36.662008341s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960872922 (253h36m36.662465364s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:34.533147917 (253h36m37.234740374s) [upload=223030]: Next returned nil segment Nov 10 01:17:34.533604854 (253h36m37.235197304s) [upload=223030]: Waiting for error group managing segments... Nov 10 01:17:34.675100228 (253h36m37.376692692s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.677791419 (253h36m37.379383873s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.917050191 (253h36m37.61864264s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.929769222 (253h36m37.631361678s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.934998499 (253h36m37.636590952s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.039420506 (253h36m37.74101296s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.051494667 (253h36m37.753087116s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.054572229 (253h36m37.756164678s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.080485645 (253h36m37.782078106s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.120581463 (253h36m37.822173915s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.212846836 (253h36m37.914439285s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:18:03.505314517 (253h37m6.206906974s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection Nov 10 01:18:03.505444022 (253h37m6.207036475s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection Nov 10 01:18:03.505769555 (253h37m6.207362014s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.505907258 (253h37m6.207499716s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection Nov 10 01:18:03.506561789 (253h37m6.208154243s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: read tcp 139.59.144.22:39162->145.79.227.16:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.506606954 (253h37m6.208199394s) [seg_pos={0 0}, upload=223030]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 10 01:18:03.619588197 (253h37m6.321180649s) [upload=223030]: Done uploading segments... Nov 10 01:18:03.619624333 (253h37m6.321216788s) [upload=223030]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.51512282s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.338208484s trace: https://jaeger.dev.storj.io/trace/3e23ce93b848e884 FAIL (2025-11-10 01:24:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 10 01:17:33.791953568 (253h36m36.493546021s) [upload=223030]: Starting upload Nov 10 01:17:33.792024560 (253h36m36.493617011s) [upload=223030]: Done starting upload Nov 10 01:17:33.792088057 (253h36m36.4936805s) [upload=223030]: Uploading segments... Nov 10 01:17:33.792513920 (253h36m36.494106378s) [upload=223030]: Got next segment. Inline: false Nov 10 01:17:33.792607652 (253h36m36.494200103s) [upload=223030]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 10 01:17:33.792615385 (253h36m36.494207835s) [upload=223030]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 10 01:17:33.904968320 (253h36m36.606560773s) [seg_pos={0 0}, upload=223030]: Begin upload segment... Nov 10 01:17:33.905008391 (253h36m36.606600841s) [seg_pos={0 0}, upload=223030]: Done begin upload segment. Nov 10 01:17:33.905031334 (253h36m36.606623791s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.905552396 (253h36m36.607144831s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.908076814 (253h36m36.609669269s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.909647138 (253h36m36.611239619s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910297142 (253h36m36.611889597s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910862225 (253h36m36.612454684s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911330437 (253h36m36.61292289s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911797731 (253h36m36.613390187s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.920749518 (253h36m36.622341975s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.930460228 (253h36m36.632052689s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.935906064 (253h36m36.637498526s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.952888411 (253h36m36.654501222s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958284571 (253h36m36.659877059s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958405123 (253h36m36.659997559s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960415899 (253h36m36.662008341s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960872922 (253h36m36.662465364s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:34.533147917 (253h36m37.234740374s) [upload=223030]: Next returned nil segment Nov 10 01:17:34.533604854 (253h36m37.235197304s) [upload=223030]: Waiting for error group managing segments... Nov 10 01:17:34.675100228 (253h36m37.376692692s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.677791419 (253h36m37.379383873s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.917050191 (253h36m37.61864264s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.929769222 (253h36m37.631361678s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.934998499 (253h36m37.636590952s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.039420506 (253h36m37.74101296s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.051494667 (253h36m37.753087116s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.054572229 (253h36m37.756164678s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.080485645 (253h36m37.782078106s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.120581463 (253h36m37.822173915s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.212846836 (253h36m37.914439285s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:18:03.505314517 (253h37m6.206906974s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection Nov 10 01:18:03.505444022 (253h37m6.207036475s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection Nov 10 01:18:03.505769555 (253h37m6.207362014s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.505907258 (253h37m6.207499716s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection Nov 10 01:18:03.506561789 (253h37m6.208154243s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: read tcp 139.59.144.22:39162->145.79.227.16:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.506606954 (253h37m6.208199394s) [seg_pos={0 0}, upload=223030]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 10 01:18:03.619588197 (253h37m6.321180649s) [upload=223030]: Done uploading segments... Nov 10 01:18:03.619624333 (253h37m6.321216788s) [upload=223030]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.338234751s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.2931106s trace: https://jaeger.dev.storj.io/trace/3e23ce93b848e884 FAIL (2025-11-10 01:24:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 10 01:17:33.791953568 (253h36m36.493546021s) [upload=223030]: Starting upload Nov 10 01:17:33.792024560 (253h36m36.493617011s) [upload=223030]: Done starting upload Nov 10 01:17:33.792088057 (253h36m36.4936805s) [upload=223030]: Uploading segments... Nov 10 01:17:33.792513920 (253h36m36.494106378s) [upload=223030]: Got next segment. Inline: false Nov 10 01:17:33.792607652 (253h36m36.494200103s) [upload=223030]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 10 01:17:33.792615385 (253h36m36.494207835s) [upload=223030]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 10 01:17:33.904968320 (253h36m36.606560773s) [seg_pos={0 0}, upload=223030]: Begin upload segment... Nov 10 01:17:33.905008391 (253h36m36.606600841s) [seg_pos={0 0}, upload=223030]: Done begin upload segment. Nov 10 01:17:33.905031334 (253h36m36.606623791s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.905552396 (253h36m36.607144831s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.908076814 (253h36m36.609669269s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.909647138 (253h36m36.611239619s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910297142 (253h36m36.611889597s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910862225 (253h36m36.612454684s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911330437 (253h36m36.61292289s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911797731 (253h36m36.613390187s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.920749518 (253h36m36.622341975s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.930460228 (253h36m36.632052689s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.935906064 (253h36m36.637498526s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.952888411 (253h36m36.654501222s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958284571 (253h36m36.659877059s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958405123 (253h36m36.659997559s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960415899 (253h36m36.662008341s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960872922 (253h36m36.662465364s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:34.533147917 (253h36m37.234740374s) [upload=223030]: Next returned nil segment Nov 10 01:17:34.533604854 (253h36m37.235197304s) [upload=223030]: Waiting for error group managing segments... Nov 10 01:17:34.675100228 (253h36m37.376692692s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.677791419 (253h36m37.379383873s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.917050191 (253h36m37.61864264s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.929769222 (253h36m37.631361678s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.934998499 (253h36m37.636590952s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.039420506 (253h36m37.74101296s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.051494667 (253h36m37.753087116s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.054572229 (253h36m37.756164678s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.080485645 (253h36m37.782078106s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.120581463 (253h36m37.822173915s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.212846836 (253h36m37.914439285s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:18:03.505314517 (253h37m6.206906974s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection Nov 10 01:18:03.505444022 (253h37m6.207036475s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection Nov 10 01:18:03.505769555 (253h37m6.207362014s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.505907258 (253h37m6.207499716s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection Nov 10 01:18:03.506561789 (253h37m6.208154243s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: read tcp 139.59.144.22:39162->145.79.227.16:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.506606954 (253h37m6.208199394s) [seg_pos={0 0}, upload=223030]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 10 01:18:03.619588197 (253h37m6.321180649s) [upload=223030]: Done uploading segments... Nov 10 01:18:03.619624333 (253h37m6.321216788s) [upload=223030]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.293176113s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.194446818s trace: https://jaeger.dev.storj.io/trace/3e23ce93b848e884 FAIL (2025-11-10 01:23:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 10 01:17:33.791953568 (253h36m36.493546021s) [upload=223030]: Starting upload Nov 10 01:17:33.792024560 (253h36m36.493617011s) [upload=223030]: Done starting upload Nov 10 01:17:33.792088057 (253h36m36.4936805s) [upload=223030]: Uploading segments... Nov 10 01:17:33.792513920 (253h36m36.494106378s) [upload=223030]: Got next segment. Inline: false Nov 10 01:17:33.792607652 (253h36m36.494200103s) [upload=223030]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 10 01:17:33.792615385 (253h36m36.494207835s) [upload=223030]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 10 01:17:33.904968320 (253h36m36.606560773s) [seg_pos={0 0}, upload=223030]: Begin upload segment... Nov 10 01:17:33.905008391 (253h36m36.606600841s) [seg_pos={0 0}, upload=223030]: Done begin upload segment. Nov 10 01:17:33.905031334 (253h36m36.606623791s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.905552396 (253h36m36.607144831s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.908076814 (253h36m36.609669269s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.909647138 (253h36m36.611239619s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910297142 (253h36m36.611889597s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910862225 (253h36m36.612454684s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911330437 (253h36m36.61292289s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911797731 (253h36m36.613390187s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.920749518 (253h36m36.622341975s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.930460228 (253h36m36.632052689s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.935906064 (253h36m36.637498526s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.952888411 (253h36m36.654501222s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958284571 (253h36m36.659877059s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958405123 (253h36m36.659997559s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960415899 (253h36m36.662008341s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960872922 (253h36m36.662465364s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:34.533147917 (253h36m37.234740374s) [upload=223030]: Next returned nil segment Nov 10 01:17:34.533604854 (253h36m37.235197304s) [upload=223030]: Waiting for error group managing segments... Nov 10 01:17:34.675100228 (253h36m37.376692692s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.677791419 (253h36m37.379383873s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.917050191 (253h36m37.61864264s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.929769222 (253h36m37.631361678s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.934998499 (253h36m37.636590952s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.039420506 (253h36m37.74101296s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.051494667 (253h36m37.753087116s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.054572229 (253h36m37.756164678s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.080485645 (253h36m37.782078106s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.120581463 (253h36m37.822173915s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.212846836 (253h36m37.914439285s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:18:03.505314517 (253h37m6.206906974s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection Nov 10 01:18:03.505444022 (253h37m6.207036475s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection Nov 10 01:18:03.505769555 (253h37m6.207362014s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.505907258 (253h37m6.207499716s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection Nov 10 01:18:03.506561789 (253h37m6.208154243s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: read tcp 139.59.144.22:39162->145.79.227.16:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.506606954 (253h37m6.208199394s) [seg_pos={0 0}, upload=223030]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 10 01:18:03.619588197 (253h37m6.321180649s) [upload=223030]: Done uploading segments... Nov 10 01:18:03.619624333 (253h37m6.321216788s) [upload=223030]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.194493958s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.215139539s trace: https://jaeger.dev.storj.io/trace/3e23ce93b848e884 FAIL (2025-11-10 01:23:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 10 01:17:33.791953568 (253h36m36.493546021s) [upload=223030]: Starting upload Nov 10 01:17:33.792024560 (253h36m36.493617011s) [upload=223030]: Done starting upload Nov 10 01:17:33.792088057 (253h36m36.4936805s) [upload=223030]: Uploading segments... Nov 10 01:17:33.792513920 (253h36m36.494106378s) [upload=223030]: Got next segment. Inline: false Nov 10 01:17:33.792607652 (253h36m36.494200103s) [upload=223030]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 10 01:17:33.792615385 (253h36m36.494207835s) [upload=223030]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 10 01:17:33.904968320 (253h36m36.606560773s) [seg_pos={0 0}, upload=223030]: Begin upload segment... Nov 10 01:17:33.905008391 (253h36m36.606600841s) [seg_pos={0 0}, upload=223030]: Done begin upload segment. Nov 10 01:17:33.905031334 (253h36m36.606623791s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.905552396 (253h36m36.607144831s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.908076814 (253h36m36.609669269s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.909647138 (253h36m36.611239619s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910297142 (253h36m36.611889597s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910862225 (253h36m36.612454684s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911330437 (253h36m36.61292289s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911797731 (253h36m36.613390187s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.920749518 (253h36m36.622341975s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.930460228 (253h36m36.632052689s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.935906064 (253h36m36.637498526s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.952888411 (253h36m36.654501222s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958284571 (253h36m36.659877059s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958405123 (253h36m36.659997559s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960415899 (253h36m36.662008341s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960872922 (253h36m36.662465364s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:34.533147917 (253h36m37.234740374s) [upload=223030]: Next returned nil segment Nov 10 01:17:34.533604854 (253h36m37.235197304s) [upload=223030]: Waiting for error group managing segments... Nov 10 01:17:34.675100228 (253h36m37.376692692s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.677791419 (253h36m37.379383873s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.917050191 (253h36m37.61864264s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.929769222 (253h36m37.631361678s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.934998499 (253h36m37.636590952s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.039420506 (253h36m37.74101296s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.051494667 (253h36m37.753087116s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.054572229 (253h36m37.756164678s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.080485645 (253h36m37.782078106s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.120581463 (253h36m37.822173915s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.212846836 (253h36m37.914439285s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:18:03.505314517 (253h37m6.206906974s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection Nov 10 01:18:03.505444022 (253h37m6.207036475s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection Nov 10 01:18:03.505769555 (253h37m6.207362014s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.505907258 (253h37m6.207499716s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection Nov 10 01:18:03.506561789 (253h37m6.208154243s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: read tcp 139.59.144.22:39162->145.79.227.16:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.506606954 (253h37m6.208199394s) [seg_pos={0 0}, upload=223030]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 10 01:18:03.619588197 (253h37m6.321180649s) [upload=223030]: Done uploading segments... Nov 10 01:18:03.619624333 (253h37m6.321216788s) [upload=223030]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.215171175s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.364910807s trace: https://jaeger.dev.storj.io/trace/3e23ce93b848e884 FAIL (2025-11-10 01:22:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 10 01:17:33.791953568 (253h36m36.493546021s) [upload=223030]: Starting upload Nov 10 01:17:33.792024560 (253h36m36.493617011s) [upload=223030]: Done starting upload Nov 10 01:17:33.792088057 (253h36m36.4936805s) [upload=223030]: Uploading segments... Nov 10 01:17:33.792513920 (253h36m36.494106378s) [upload=223030]: Got next segment. Inline: false Nov 10 01:17:33.792607652 (253h36m36.494200103s) [upload=223030]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 10 01:17:33.792615385 (253h36m36.494207835s) [upload=223030]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 10 01:17:33.904968320 (253h36m36.606560773s) [seg_pos={0 0}, upload=223030]: Begin upload segment... Nov 10 01:17:33.905008391 (253h36m36.606600841s) [seg_pos={0 0}, upload=223030]: Done begin upload segment. Nov 10 01:17:33.905031334 (253h36m36.606623791s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.905552396 (253h36m36.607144831s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.908076814 (253h36m36.609669269s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.909647138 (253h36m36.611239619s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910297142 (253h36m36.611889597s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910862225 (253h36m36.612454684s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911330437 (253h36m36.61292289s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911797731 (253h36m36.613390187s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.920749518 (253h36m36.622341975s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.930460228 (253h36m36.632052689s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.935906064 (253h36m36.637498526s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.952888411 (253h36m36.654501222s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958284571 (253h36m36.659877059s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958405123 (253h36m36.659997559s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960415899 (253h36m36.662008341s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960872922 (253h36m36.662465364s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:34.533147917 (253h36m37.234740374s) [upload=223030]: Next returned nil segment Nov 10 01:17:34.533604854 (253h36m37.235197304s) [upload=223030]: Waiting for error group managing segments... Nov 10 01:17:34.675100228 (253h36m37.376692692s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.677791419 (253h36m37.379383873s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.917050191 (253h36m37.61864264s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.929769222 (253h36m37.631361678s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.934998499 (253h36m37.636590952s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.039420506 (253h36m37.74101296s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.051494667 (253h36m37.753087116s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.054572229 (253h36m37.756164678s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.080485645 (253h36m37.782078106s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.120581463 (253h36m37.822173915s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.212846836 (253h36m37.914439285s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:18:03.505314517 (253h37m6.206906974s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection Nov 10 01:18:03.505444022 (253h37m6.207036475s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection Nov 10 01:18:03.505769555 (253h37m6.207362014s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.505907258 (253h37m6.207499716s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection Nov 10 01:18:03.506561789 (253h37m6.208154243s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: read tcp 139.59.144.22:39162->145.79.227.16:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.506606954 (253h37m6.208199394s) [seg_pos={0 0}, upload=223030]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 10 01:18:03.619588197 (253h37m6.321180649s) [upload=223030]: Done uploading segments... Nov 10 01:18:03.619624333 (253h37m6.321216788s) [upload=223030]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.364963567s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.307827481s trace: https://jaeger.dev.storj.io/trace/3e23ce93b848e884 FAIL (2025-11-10 01:21:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 10 01:17:33.791953568 (253h36m36.493546021s) [upload=223030]: Starting upload Nov 10 01:17:33.792024560 (253h36m36.493617011s) [upload=223030]: Done starting upload Nov 10 01:17:33.792088057 (253h36m36.4936805s) [upload=223030]: Uploading segments... Nov 10 01:17:33.792513920 (253h36m36.494106378s) [upload=223030]: Got next segment. Inline: false Nov 10 01:17:33.792607652 (253h36m36.494200103s) [upload=223030]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 10 01:17:33.792615385 (253h36m36.494207835s) [upload=223030]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 10 01:17:33.904968320 (253h36m36.606560773s) [seg_pos={0 0}, upload=223030]: Begin upload segment... Nov 10 01:17:33.905008391 (253h36m36.606600841s) [seg_pos={0 0}, upload=223030]: Done begin upload segment. Nov 10 01:17:33.905031334 (253h36m36.606623791s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.905552396 (253h36m36.607144831s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.908076814 (253h36m36.609669269s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.909647138 (253h36m36.611239619s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910297142 (253h36m36.611889597s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910862225 (253h36m36.612454684s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911330437 (253h36m36.61292289s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911797731 (253h36m36.613390187s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.920749518 (253h36m36.622341975s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.930460228 (253h36m36.632052689s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.935906064 (253h36m36.637498526s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.952888411 (253h36m36.654501222s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958284571 (253h36m36.659877059s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958405123 (253h36m36.659997559s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960415899 (253h36m36.662008341s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960872922 (253h36m36.662465364s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:34.533147917 (253h36m37.234740374s) [upload=223030]: Next returned nil segment Nov 10 01:17:34.533604854 (253h36m37.235197304s) [upload=223030]: Waiting for error group managing segments... Nov 10 01:17:34.675100228 (253h36m37.376692692s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.677791419 (253h36m37.379383873s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.917050191 (253h36m37.61864264s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.929769222 (253h36m37.631361678s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.934998499 (253h36m37.636590952s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.039420506 (253h36m37.74101296s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.051494667 (253h36m37.753087116s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.054572229 (253h36m37.756164678s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.080485645 (253h36m37.782078106s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.120581463 (253h36m37.822173915s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.212846836 (253h36m37.914439285s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:18:03.505314517 (253h37m6.206906974s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection Nov 10 01:18:03.505444022 (253h37m6.207036475s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection Nov 10 01:18:03.505769555 (253h37m6.207362014s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.505907258 (253h37m6.207499716s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection Nov 10 01:18:03.506561789 (253h37m6.208154243s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: read tcp 139.59.144.22:39162->145.79.227.16:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.506606954 (253h37m6.208199394s) [seg_pos={0 0}, upload=223030]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 10 01:18:03.619588197 (253h37m6.321180649s) [upload=223030]: Done uploading segments... Nov 10 01:18:03.619624333 (253h37m6.321216788s) [upload=223030]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.307864544s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.294686895s trace: https://jaeger.dev.storj.io/trace/3e23ce93b848e884 FAIL (2025-11-10 01:21:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 10 01:17:33.791953568 (253h36m36.493546021s) [upload=223030]: Starting upload Nov 10 01:17:33.792024560 (253h36m36.493617011s) [upload=223030]: Done starting upload Nov 10 01:17:33.792088057 (253h36m36.4936805s) [upload=223030]: Uploading segments... Nov 10 01:17:33.792513920 (253h36m36.494106378s) [upload=223030]: Got next segment. Inline: false Nov 10 01:17:33.792607652 (253h36m36.494200103s) [upload=223030]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 10 01:17:33.792615385 (253h36m36.494207835s) [upload=223030]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 10 01:17:33.904968320 (253h36m36.606560773s) [seg_pos={0 0}, upload=223030]: Begin upload segment... Nov 10 01:17:33.905008391 (253h36m36.606600841s) [seg_pos={0 0}, upload=223030]: Done begin upload segment. Nov 10 01:17:33.905031334 (253h36m36.606623791s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.905552396 (253h36m36.607144831s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.908076814 (253h36m36.609669269s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.909647138 (253h36m36.611239619s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910297142 (253h36m36.611889597s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910862225 (253h36m36.612454684s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911330437 (253h36m36.61292289s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911797731 (253h36m36.613390187s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.920749518 (253h36m36.622341975s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.930460228 (253h36m36.632052689s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.935906064 (253h36m36.637498526s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.952888411 (253h36m36.654501222s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958284571 (253h36m36.659877059s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958405123 (253h36m36.659997559s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960415899 (253h36m36.662008341s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960872922 (253h36m36.662465364s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:34.533147917 (253h36m37.234740374s) [upload=223030]: Next returned nil segment Nov 10 01:17:34.533604854 (253h36m37.235197304s) [upload=223030]: Waiting for error group managing segments... Nov 10 01:17:34.675100228 (253h36m37.376692692s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.677791419 (253h36m37.379383873s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.917050191 (253h36m37.61864264s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.929769222 (253h36m37.631361678s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.934998499 (253h36m37.636590952s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.039420506 (253h36m37.74101296s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.051494667 (253h36m37.753087116s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.054572229 (253h36m37.756164678s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.080485645 (253h36m37.782078106s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.120581463 (253h36m37.822173915s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.212846836 (253h36m37.914439285s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:18:03.505314517 (253h37m6.206906974s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection Nov 10 01:18:03.505444022 (253h37m6.207036475s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection Nov 10 01:18:03.505769555 (253h37m6.207362014s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.505907258 (253h37m6.207499716s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection Nov 10 01:18:03.506561789 (253h37m6.208154243s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: read tcp 139.59.144.22:39162->145.79.227.16:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.506606954 (253h37m6.208199394s) [seg_pos={0 0}, upload=223030]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 10 01:18:03.619588197 (253h37m6.321180649s) [upload=223030]: Done uploading segments... Nov 10 01:18:03.619624333 (253h37m6.321216788s) [upload=223030]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.294752924s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.97063163s trace: https://jaeger.dev.storj.io/trace/3e23ce93b848e884 FAIL (2025-11-10 01:20:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 10 01:17:33.791953568 (253h36m36.493546021s) [upload=223030]: Starting upload Nov 10 01:17:33.792024560 (253h36m36.493617011s) [upload=223030]: Done starting upload Nov 10 01:17:33.792088057 (253h36m36.4936805s) [upload=223030]: Uploading segments... Nov 10 01:17:33.792513920 (253h36m36.494106378s) [upload=223030]: Got next segment. Inline: false Nov 10 01:17:33.792607652 (253h36m36.494200103s) [upload=223030]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 10 01:17:33.792615385 (253h36m36.494207835s) [upload=223030]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 10 01:17:33.904968320 (253h36m36.606560773s) [seg_pos={0 0}, upload=223030]: Begin upload segment... Nov 10 01:17:33.905008391 (253h36m36.606600841s) [seg_pos={0 0}, upload=223030]: Done begin upload segment. Nov 10 01:17:33.905031334 (253h36m36.606623791s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.905552396 (253h36m36.607144831s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.908076814 (253h36m36.609669269s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.909647138 (253h36m36.611239619s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910297142 (253h36m36.611889597s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910862225 (253h36m36.612454684s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911330437 (253h36m36.61292289s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911797731 (253h36m36.613390187s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.920749518 (253h36m36.622341975s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.930460228 (253h36m36.632052689s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.935906064 (253h36m36.637498526s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.952888411 (253h36m36.654501222s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958284571 (253h36m36.659877059s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958405123 (253h36m36.659997559s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960415899 (253h36m36.662008341s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960872922 (253h36m36.662465364s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:34.533147917 (253h36m37.234740374s) [upload=223030]: Next returned nil segment Nov 10 01:17:34.533604854 (253h36m37.235197304s) [upload=223030]: Waiting for error group managing segments... Nov 10 01:17:34.675100228 (253h36m37.376692692s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.677791419 (253h36m37.379383873s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.917050191 (253h36m37.61864264s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.929769222 (253h36m37.631361678s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.934998499 (253h36m37.636590952s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.039420506 (253h36m37.74101296s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.051494667 (253h36m37.753087116s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.054572229 (253h36m37.756164678s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.080485645 (253h36m37.782078106s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.120581463 (253h36m37.822173915s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.212846836 (253h36m37.914439285s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:18:03.505314517 (253h37m6.206906974s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection Nov 10 01:18:03.505444022 (253h37m6.207036475s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection Nov 10 01:18:03.505769555 (253h37m6.207362014s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.505907258 (253h37m6.207499716s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection Nov 10 01:18:03.506561789 (253h37m6.208154243s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: read tcp 139.59.144.22:39162->145.79.227.16:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.506606954 (253h37m6.208199394s) [seg_pos={0 0}, upload=223030]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 10 01:18:03.619588197 (253h37m6.321180649s) [upload=223030]: Done uploading segments... Nov 10 01:18:03.619624333 (253h37m6.321216788s) [upload=223030]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.970730168s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.38663373s trace: https://jaeger.dev.storj.io/trace/3e23ce93b848e884 FAIL (2025-11-10 01:18:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 10 01:17:33.791953568 (253h36m36.493546021s) [upload=223030]: Starting upload Nov 10 01:17:33.792024560 (253h36m36.493617011s) [upload=223030]: Done starting upload Nov 10 01:17:33.792088057 (253h36m36.4936805s) [upload=223030]: Uploading segments... Nov 10 01:17:33.792513920 (253h36m36.494106378s) [upload=223030]: Got next segment. Inline: false Nov 10 01:17:33.792607652 (253h36m36.494200103s) [upload=223030]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 10 01:17:33.792615385 (253h36m36.494207835s) [upload=223030]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 10 01:17:33.904968320 (253h36m36.606560773s) [seg_pos={0 0}, upload=223030]: Begin upload segment... Nov 10 01:17:33.905008391 (253h36m36.606600841s) [seg_pos={0 0}, upload=223030]: Done begin upload segment. Nov 10 01:17:33.905031334 (253h36m36.606623791s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.905552396 (253h36m36.607144831s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.908076814 (253h36m36.609669269s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.909647138 (253h36m36.611239619s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910297142 (253h36m36.611889597s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910862225 (253h36m36.612454684s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911330437 (253h36m36.61292289s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911797731 (253h36m36.613390187s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.920749518 (253h36m36.622341975s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.930460228 (253h36m36.632052689s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.935906064 (253h36m36.637498526s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.952888411 (253h36m36.654501222s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958284571 (253h36m36.659877059s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958405123 (253h36m36.659997559s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960415899 (253h36m36.662008341s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960872922 (253h36m36.662465364s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:34.533147917 (253h36m37.234740374s) [upload=223030]: Next returned nil segment Nov 10 01:17:34.533604854 (253h36m37.235197304s) [upload=223030]: Waiting for error group managing segments... Nov 10 01:17:34.675100228 (253h36m37.376692692s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.677791419 (253h36m37.379383873s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.917050191 (253h36m37.61864264s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.929769222 (253h36m37.631361678s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.934998499 (253h36m37.636590952s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.039420506 (253h36m37.74101296s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.051494667 (253h36m37.753087116s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.054572229 (253h36m37.756164678s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.080485645 (253h36m37.782078106s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.120581463 (253h36m37.822173915s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.212846836 (253h36m37.914439285s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:18:03.505314517 (253h37m6.206906974s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection Nov 10 01:18:03.505444022 (253h37m6.207036475s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection Nov 10 01:18:03.505769555 (253h37m6.207362014s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.505907258 (253h37m6.207499716s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection Nov 10 01:18:03.506561789 (253h37m6.208154243s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: read tcp 139.59.144.22:39162->145.79.227.16:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.506606954 (253h37m6.208199394s) [seg_pos={0 0}, upload=223030]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 10 01:18:03.619588197 (253h37m6.321180649s) [upload=223030]: Done uploading segments... Nov 10 01:18:03.619624333 (253h37m6.321216788s) [upload=223030]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.386666565s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.11782443s trace: https://jaeger.dev.storj.io/trace/3e23ce93b848e884 FAIL (2025-11-10 01:18:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 10 01:17:33.791953568 (253h36m36.493546021s) [upload=223030]: Starting upload Nov 10 01:17:33.792024560 (253h36m36.493617011s) [upload=223030]: Done starting upload Nov 10 01:17:33.792088057 (253h36m36.4936805s) [upload=223030]: Uploading segments... Nov 10 01:17:33.792513920 (253h36m36.494106378s) [upload=223030]: Got next segment. Inline: false Nov 10 01:17:33.792607652 (253h36m36.494200103s) [upload=223030]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 10 01:17:33.792615385 (253h36m36.494207835s) [upload=223030]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 10 01:17:33.904968320 (253h36m36.606560773s) [seg_pos={0 0}, upload=223030]: Begin upload segment... Nov 10 01:17:33.905008391 (253h36m36.606600841s) [seg_pos={0 0}, upload=223030]: Done begin upload segment. Nov 10 01:17:33.905031334 (253h36m36.606623791s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.905552396 (253h36m36.607144831s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.908076814 (253h36m36.609669269s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.909647138 (253h36m36.611239619s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910297142 (253h36m36.611889597s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.910862225 (253h36m36.612454684s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911330437 (253h36m36.61292289s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.911797731 (253h36m36.613390187s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.920749518 (253h36m36.622341975s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.930460228 (253h36m36.632052689s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.935906064 (253h36m36.637498526s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.952888411 (253h36m36.654501222s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958284571 (253h36m36.659877059s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.958405123 (253h36m36.659997559s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960415899 (253h36m36.662008341s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:33.960872922 (253h36m36.662465364s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Uploading piece... Nov 10 01:17:34.533147917 (253h36m37.234740374s) [upload=223030]: Next returned nil segment Nov 10 01:17:34.533604854 (253h36m37.235197304s) [upload=223030]: Waiting for error group managing segments... Nov 10 01:17:34.675100228 (253h36m37.376692692s) [noise=true , address=77.90.31.14:4663 , piece_id=YUNIDFZ27NPAYT6T7KYIFYPBUOC76YTPVHE6FK5A5Q76FIHKH7XA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.677791419 (253h36m37.379383873s) [noise=true , address=45.91.116.4:4939 , piece_id=IVZZZG5APPATBDSR2IC7MM25N2DERPIQYWJ2QOXAENZNVPEZW5AQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.917050191 (253h36m37.61864264s) [noise=true , address=145.79.227.8:4331 , piece_id=E7W6LR35ZQEVFASKWKGYPLH4YYX6H3SEYQNJHWXLAT6Q4AHHNXNQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.929769222 (253h36m37.631361678s) [noise=true , address=145.79.227.31:4799 , piece_id=ZNOAAIDZV3CV2PALHERDIFE3XOQVVTH5GZAGWIINZBNDEKJVG2WA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:34.934998499 (253h36m37.636590952s) [noise=true , address=145.79.227.30:4763 , piece_id=QCTLOXH2WDVJUY3T535VBT4CX7OOEXN2DUANIG274BZIU2F4ZHUA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.039420506 (253h36m37.74101296s) [noise=true , address=145.79.227.13:4802 , piece_id=6HS7F3EF7RJGST4CYVMFQJDAWH7F27WNEJECZ3TB335H2MQGV2KA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.051494667 (253h36m37.753087116s) [noise=true , address=145.79.227.21:4473 , piece_id=KPJ2D5AX4JRYCL673WWW6DNAUN42ZO6TKZNNWGNSZTKHJMOMBYHA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.054572229 (253h36m37.756164678s) [noise=true , address=145.79.227.32:4835 , piece_id=7S2UMZUULINY7V2GU5QOHOEPSJQDLM4XSNK4AH4QMQJKKDGJ2H4A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.080485645 (253h36m37.782078106s) [noise=true , address=145.79.227.11:4732 , piece_id=WENSMMFUUDMVJZ74ZOMHNFY75MV5XJ4HRBA25IOOBUPBBO73U4JQ, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.120581463 (253h36m37.822173915s) [noise=true , address=145.79.227.23:4546 , piece_id=6F7NGJTYFFD7RKP7IYV3YUCZWHYJ763DWA2EQ7JDEMQJCAINH3PA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:17:35.212846836 (253h36m37.914439285s) [noise=true , address=145.79.227.22:4509 , piece_id=NMDHBL53NUNBXJ2RHAFW47TKFXODWVOHB4V32JNYPGDPTWMNDG4Q, seg_pos={0 0}, upload=223030]: Done uploading piece. err: <nil> Nov 10 01:18:03.505314517 (253h37m6.206906974s) [noise=true , address=145.79.227.7:4294 , piece_id=3HUZCMUQ5UXISA2A4V5BQAE4RUAIJWOQZYZJKKDEA3JMGXPJMOPA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37250->145.79.227.7:4294: use of closed network connection Nov 10 01:18:03.505444022 (253h37m6.207036475s) [noise=true , address=145.79.227.6:4258 , piece_id=ERCELQ25ETSEP4RGBNDAROHVMT43YKGQ3GGKPYYBRX4JR5C3HN3A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44290->145.79.227.6:4258: use of closed network connection Nov 10 01:18:03.505769555 (253h37m6.207362014s) [noise=true , address=145.79.227.17:4970 , piece_id=2FGBMRJWEUCO5SXYLDBVQTC7T7CHQQA5SIFBXZB3EDDLXR7ME3AA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.505907258 (253h37m6.207499716s) [noise=true , address=145.79.227.24:4583 , piece_id=EZPCFHRLHPWY6EUIHHPJQ26MH4PQV3AP7B75LLJL7LE3EHFEHMRA, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54726->145.79.227.24:4583: use of closed network connection Nov 10 01:18:03.506561789 (253h37m6.208154243s) [noise=true , address=145.79.227.16:4911 , piece_id=CPI5AY5KIIK7T7RW7IXWTNN3JGGPHSXX3HJDJVECM5KKN5TY763A, seg_pos={0 0}, upload=223030]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: read tcp 139.59.144.22:39162->145.79.227.16:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 10 01:18:03.506606954 (253h37m6.208199394s) [seg_pos={0 0}, upload=223030]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 10 01:18:03.619588197 (253h37m6.321180649s) [upload=223030]: Done uploading segments... Nov 10 01:18:03.619624333 (253h37m6.321216788s) [upload=223030]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.117847548s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [X...........................................................] duration: 678.250545ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:22:35 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 678.276975ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.X..........................................................] duration: 238.287298ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:21:54 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 238.31727ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..X.........................................................] duration: 234.670029ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:20:52 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 234.72958ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...X........................................................] duration: 269.06704ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:20:25 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 269.114084ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [....X.......................................................] duration: 689.301614ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:19:22 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 689.400439ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.....X......................................................] duration: 246.805684ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:18:47 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 246.836544ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [......X.....................................................] duration: 253.020552ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:17:47 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 253.045451ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.......X....................................................] duration: 261.856155ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:16:48 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 261.88734ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [........X...................................................] duration: 260.488043ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:16:32 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 260.523757ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.........X..................................................] duration: 246.713859ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:16:06 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 246.743293ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..........X.................................................] duration: 6.128188655s remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:15:34 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 6.128219205s edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...........X................................................] duration: 244.090742ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:15:12 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 244.150774ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [............X...............................................] duration: 240.145032ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:14:28 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 240.173244ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.............X..............................................] duration: 678.182937ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:13:28 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 678.281966ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..............X.............................................] duration: 244.226806ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:12:41 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 244.319928ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...............X............................................] duration: 701.430357ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:11:57 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 701.461011ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [................X...........................................] duration: 246.68826ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:10:49 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 246.741095ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.................X..........................................] duration: 665.404855ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:09:35 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 665.522946ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..................X.........................................] duration: 205.646309ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:08:39 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 205.666373ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...................X........................................] duration: 1.343555028s remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:07:58 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 1.343596409s edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [....................X.......................................] duration: 571.323921ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:07:21 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 571.364195ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.....................X......................................] duration: 195.804818ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:06:53 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 195.859346ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [......................X.....................................] duration: 3.246000133s remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:05:54 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 3.246029058s edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.......................X....................................] duration: 1.275771504s remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:04:47 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 1.275794936s edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [........................X...................................] duration: 238.470217ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:03:46 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 238.561558ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.........................X..................................] duration: 241.286325ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:03:20 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 241.322574ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..........................X.................................] duration: 231.073419ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:02:25 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 231.097754ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...........................X................................] duration: 43.506668492s remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:02:09 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 43.506712748s edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [............................X...............................] duration: 260.268025ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 19:00:27 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 260.30809ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.............................X..............................] duration: 678.286512ms remote: "tcp:185.244.226.3:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/203ee1813906a750 FAIL (2025-11-09 18:59:31 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 678.321675ms edge linksharing bgp ap1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)