Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.837991662s FAIL (2025-07-18 11:27:04 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.838034571s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......................X.....X.X..........X...............X.] duration: 2.374526299s FAIL (2025-07-18 11:26:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 139.59.144.22:52386->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:25:03.756616000 (602h14m3.640029628s) [seg_pos={0 0}, upload=481509]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:25:03.756750542 (602h14m3.640164173s) [upload=481509]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:25:03.756757773 (602h14m3.640171571s) [upload=481509]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:25:03.918610530 (602h14m3.802024164s) [upload=481509]: Done uploading segments... Jul 18 11:25:03.918632896 (602h14m3.802046512s) [upload=481509]: Upload finished. err: <nil> Jul 18 11:25:56.053888090 (602h14m55.937301716s) [upload=481521]: Starting upload Jul 18 11:25:56.053994103 (602h14m55.937407737s) [upload=481521]: Done starting upload Jul 18 11:25:56.054129122 (602h14m55.937542749s) [upload=481521]: Uploading segments... Jul 18 11:25:56.054306891 (602h14m55.937720517s) [upload=481521]: Got next segment. Inline: false Jul 18 11:25:56.054364770 (602h14m55.937778397s) [upload=481521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:25:56.054371001 (602h14m55.937784628s) [upload=481521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:25:56.170318697 (602h14m56.053732323s) [seg_pos={0 0}, upload=481521]: Begin upload segment... Jul 18 11:25:56.170422900 (602h14m56.053836526s) [seg_pos={0 0}, upload=481521]: Done begin upload segment. Jul 18 11:25:56.170464681 (602h14m56.053878314s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170521320 (602h14m56.053934956s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170853558 (602h14m56.054267185s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170909345 (602h14m56.054322977s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171219719 (602h14m56.054633352s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171337279 (602h14m56.054750915s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171541852 (602h14m56.054955482s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171704517 (602h14m56.055118151s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171920013 (602h14m56.055333643s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172277758 (602h14m56.055691391s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172326432 (602h14m56.055740064s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172568781 (602h14m56.055982394s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172668061 (602h14m56.056081694s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172832081 (602h14m56.056245713s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173001844 (602h14m56.056415473s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173339770 (602h14m56.056753404s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.786495143 (602h14m56.669908759s) [upload=481521]: Next returned nil segment Jul 18 11:25:56.786512550 (602h14m56.669926162s) [upload=481521]: Waiting for error group managing segments... Jul 18 11:25:56.918094694 (602h14m56.80150833s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.918628075 (602h14m56.802041694s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.919912323 (602h14m56.803325943s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.920806034 (602h14m56.804219647s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.921420465 (602h14m56.804834084s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.923349719 (602h14m56.80676334s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.926920740 (602h14m56.810334354s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.930546114 (602h14m56.813959734s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.931434176 (602h14m56.814847796s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.945660373 (602h14m56.829074004s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.955452695 (602h14m56.838866315s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.957436329 (602h14m56.840849962s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:26:25.766597801 (602h15m25.65001142s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766639011 (602h15m25.650052626s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:53448->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766658832 (602h15m25.650072444s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766671419 (602h15m25.650085044s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:48546->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766698381 (602h15m25.650112007s) [seg_pos={0 0}, upload=481521]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 11:26:25.884294106 (602h15m25.76770774s) [upload=481521]: Done uploading segments... Jul 18 11:26:25.884340118 (602h15m25.767753759s) [upload=481521]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.374558069s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 1.142221701s FAIL (2025-07-18 11:26:54 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.142267941s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.507404026s FAIL (2025-07-18 11:26:45 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.507436273s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 1.082733389s FAIL (2025-07-18 11:26:26 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.082753943s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [........................X.....X.X..........X...............X] duration: 30.119149674s FAIL (2025-07-18 11:26:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 139.59.144.22:52386->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:25:03.756616000 (602h14m3.640029628s) [seg_pos={0 0}, upload=481509]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:25:03.756750542 (602h14m3.640164173s) [upload=481509]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:25:03.756757773 (602h14m3.640171571s) [upload=481509]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:25:03.918610530 (602h14m3.802024164s) [upload=481509]: Done uploading segments... Jul 18 11:25:03.918632896 (602h14m3.802046512s) [upload=481509]: Upload finished. err: <nil> Jul 18 11:25:56.053888090 (602h14m55.937301716s) [upload=481521]: Starting upload Jul 18 11:25:56.053994103 (602h14m55.937407737s) [upload=481521]: Done starting upload Jul 18 11:25:56.054129122 (602h14m55.937542749s) [upload=481521]: Uploading segments... Jul 18 11:25:56.054306891 (602h14m55.937720517s) [upload=481521]: Got next segment. Inline: false Jul 18 11:25:56.054364770 (602h14m55.937778397s) [upload=481521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:25:56.054371001 (602h14m55.937784628s) [upload=481521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:25:56.170318697 (602h14m56.053732323s) [seg_pos={0 0}, upload=481521]: Begin upload segment... Jul 18 11:25:56.170422900 (602h14m56.053836526s) [seg_pos={0 0}, upload=481521]: Done begin upload segment. Jul 18 11:25:56.170464681 (602h14m56.053878314s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170521320 (602h14m56.053934956s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170853558 (602h14m56.054267185s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170909345 (602h14m56.054322977s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171219719 (602h14m56.054633352s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171337279 (602h14m56.054750915s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171541852 (602h14m56.054955482s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171704517 (602h14m56.055118151s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171920013 (602h14m56.055333643s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172277758 (602h14m56.055691391s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172326432 (602h14m56.055740064s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172568781 (602h14m56.055982394s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172668061 (602h14m56.056081694s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172832081 (602h14m56.056245713s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173001844 (602h14m56.056415473s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173339770 (602h14m56.056753404s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.786495143 (602h14m56.669908759s) [upload=481521]: Next returned nil segment Jul 18 11:25:56.786512550 (602h14m56.669926162s) [upload=481521]: Waiting for error group managing segments... Jul 18 11:25:56.918094694 (602h14m56.80150833s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.918628075 (602h14m56.802041694s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.919912323 (602h14m56.803325943s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.920806034 (602h14m56.804219647s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.921420465 (602h14m56.804834084s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.923349719 (602h14m56.80676334s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.926920740 (602h14m56.810334354s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.930546114 (602h14m56.813959734s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.931434176 (602h14m56.814847796s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.945660373 (602h14m56.829074004s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.955452695 (602h14m56.838866315s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.957436329 (602h14m56.840849962s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:26:25.766597801 (602h15m25.65001142s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766639011 (602h15m25.650052626s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:53448->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766658832 (602h15m25.650072444s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766671419 (602h15m25.650085044s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:48546->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766698381 (602h15m25.650112007s) [seg_pos={0 0}, upload=481521]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 11:26:25.884294106 (602h15m25.76770774s) [upload=481521]: Done uploading segments... Jul 18 11:26:25.884340118 (602h15m25.767753759s) [upload=481521]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.119175258s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.660840714s FAIL (2025-07-18 11:25:31 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.660871686s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 1.082008193s FAIL (2025-07-18 11:25:28 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.082072211s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................X.....X.X..........X...............] duration: 2.437066537s FAIL (2025-07-18 11:25:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== g for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:13:13.115856263 (602h2m12.999269893s) [upload=481356]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:13:13.115862143 (602h2m12.999275774s) [upload=481356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:13:13.309475952 (602h2m13.192889568s) [upload=481356]: Done uploading segments... Jul 18 11:13:13.309494751 (602h2m13.192908371s) [upload=481356]: Upload finished. err: <nil> Jul 18 11:14:02.324953677 (602h3m2.208367292s) [upload=481367]: Starting upload Jul 18 11:14:02.325059357 (602h3m2.208472991s) [upload=481367]: Done starting upload Jul 18 11:14:02.325168053 (602h3m2.208581686s) [upload=481367]: Uploading segments... Jul 18 11:14:02.325655202 (602h3m2.20906884s) [upload=481367]: Got next segment. Inline: false Jul 18 11:14:02.325752146 (602h3m2.20916578s) [upload=481367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:14:02.325761218 (602h3m2.209174851s) [upload=481367]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:14:02.441249374 (602h3m2.324663009s) [seg_pos={0 0}, upload=481367]: Begin upload segment... Jul 18 11:14:02.441362061 (602h3m2.324775708s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442051381 (602h3m2.325465045s) [seg_pos={0 0}, upload=481367]: Done begin upload segment. Jul 18 11:14:02.442129239 (602h3m2.325542862s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442138594 (602h3m2.325552237s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442226743 (602h3m2.325640435s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442714959 (602h3m2.326128728s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442758980 (602h3m2.326172617s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.443182422 (602h3m2.32659606s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.444366357 (602h3m2.327780009s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445017131 (602h3m2.328430777s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445631564 (602h3m2.329045207s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450384259 (602h3m2.333797901s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450888992 (602h3m2.334302623s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451286117 (602h3m2.334699756s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451731093 (602h3m2.33514473s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.452810956 (602h3m2.336224591s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.453311896 (602h3m2.336725535s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:03.061350628 (602h3m2.944764264s) [upload=481367]: Next returned nil segment Jul 18 11:14:03.061375887 (602h3m2.944789525s) [upload=481367]: Waiting for error group managing segments... Jul 18 11:14:03.200779365 (602h3m3.084192987s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.237279929 (602h3m3.120693557s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.238276497 (602h3m3.121690114s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.247803493 (602h3m3.131217125s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.249377712 (602h3m3.132791354s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.253004867 (602h3m3.136418496s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.261597239 (602h3m3.145010874s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.262347910 (602h3m3.145761594s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.291695089 (602h3m3.175108725s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:04.910060448 (602h3m4.793474069s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:32.042078624 (602h3m31.925492241s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042148842 (602h3m31.925562458s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042190707 (602h3m31.925604323s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042239456 (602h3m31.925653071s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042286864 (602h3m31.92570048s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042338367 (602h3m31.925751984s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:37634->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042355181 (602h3m31.925768813s) [seg_pos={0 0}, upload=481367]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 11:14:32.155211809 (602h3m32.038625429s) [upload=481367]: Done uploading segments... Jul 18 11:14:32.155246584 (602h3m32.038660215s) [upload=481367]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.437094542s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.672825773s FAIL (2025-07-18 11:24:47 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.672856666s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 1.089287974s FAIL (2025-07-18 11:24:45 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.089327223s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 1.127950399s FAIL (2025-07-18 11:24:20 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.127970119s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................X.....X.X..........X..............] duration: 2.473455429s FAIL (2025-07-18 11:24:16 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== g for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:13:13.115856263 (602h2m12.999269893s) [upload=481356]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:13:13.115862143 (602h2m12.999275774s) [upload=481356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:13:13.309475952 (602h2m13.192889568s) [upload=481356]: Done uploading segments... Jul 18 11:13:13.309494751 (602h2m13.192908371s) [upload=481356]: Upload finished. err: <nil> Jul 18 11:14:02.324953677 (602h3m2.208367292s) [upload=481367]: Starting upload Jul 18 11:14:02.325059357 (602h3m2.208472991s) [upload=481367]: Done starting upload Jul 18 11:14:02.325168053 (602h3m2.208581686s) [upload=481367]: Uploading segments... Jul 18 11:14:02.325655202 (602h3m2.20906884s) [upload=481367]: Got next segment. Inline: false Jul 18 11:14:02.325752146 (602h3m2.20916578s) [upload=481367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:14:02.325761218 (602h3m2.209174851s) [upload=481367]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:14:02.441249374 (602h3m2.324663009s) [seg_pos={0 0}, upload=481367]: Begin upload segment... Jul 18 11:14:02.441362061 (602h3m2.324775708s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442051381 (602h3m2.325465045s) [seg_pos={0 0}, upload=481367]: Done begin upload segment. Jul 18 11:14:02.442129239 (602h3m2.325542862s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442138594 (602h3m2.325552237s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442226743 (602h3m2.325640435s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442714959 (602h3m2.326128728s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442758980 (602h3m2.326172617s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.443182422 (602h3m2.32659606s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.444366357 (602h3m2.327780009s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445017131 (602h3m2.328430777s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445631564 (602h3m2.329045207s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450384259 (602h3m2.333797901s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450888992 (602h3m2.334302623s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451286117 (602h3m2.334699756s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451731093 (602h3m2.33514473s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.452810956 (602h3m2.336224591s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.453311896 (602h3m2.336725535s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:03.061350628 (602h3m2.944764264s) [upload=481367]: Next returned nil segment Jul 18 11:14:03.061375887 (602h3m2.944789525s) [upload=481367]: Waiting for error group managing segments... Jul 18 11:14:03.200779365 (602h3m3.084192987s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.237279929 (602h3m3.120693557s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.238276497 (602h3m3.121690114s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.247803493 (602h3m3.131217125s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.249377712 (602h3m3.132791354s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.253004867 (602h3m3.136418496s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.261597239 (602h3m3.145010874s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.262347910 (602h3m3.145761594s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.291695089 (602h3m3.175108725s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:04.910060448 (602h3m4.793474069s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:32.042078624 (602h3m31.925492241s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042148842 (602h3m31.925562458s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042190707 (602h3m31.925604323s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042239456 (602h3m31.925653071s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042286864 (602h3m31.92570048s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042338367 (602h3m31.925751984s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:37634->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042355181 (602h3m31.925768813s) [seg_pos={0 0}, upload=481367]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 11:14:32.155211809 (602h3m32.038625429s) [upload=481367]: Done uploading segments... Jul 18 11:14:32.155246584 (602h3m32.038660215s) [upload=481367]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.473487953s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.631189463s FAIL (2025-07-18 11:23:37 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.631221676s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 1.090640265s FAIL (2025-07-18 11:23:20 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.090687133s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................X.....X.X..........X.............] duration: 2.3547757s FAIL (2025-07-18 11:23:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== g for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:13:13.115856263 (602h2m12.999269893s) [upload=481356]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:13:13.115862143 (602h2m12.999275774s) [upload=481356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:13:13.309475952 (602h2m13.192889568s) [upload=481356]: Done uploading segments... Jul 18 11:13:13.309494751 (602h2m13.192908371s) [upload=481356]: Upload finished. err: <nil> Jul 18 11:14:02.324953677 (602h3m2.208367292s) [upload=481367]: Starting upload Jul 18 11:14:02.325059357 (602h3m2.208472991s) [upload=481367]: Done starting upload Jul 18 11:14:02.325168053 (602h3m2.208581686s) [upload=481367]: Uploading segments... Jul 18 11:14:02.325655202 (602h3m2.20906884s) [upload=481367]: Got next segment. Inline: false Jul 18 11:14:02.325752146 (602h3m2.20916578s) [upload=481367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:14:02.325761218 (602h3m2.209174851s) [upload=481367]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:14:02.441249374 (602h3m2.324663009s) [seg_pos={0 0}, upload=481367]: Begin upload segment... Jul 18 11:14:02.441362061 (602h3m2.324775708s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442051381 (602h3m2.325465045s) [seg_pos={0 0}, upload=481367]: Done begin upload segment. Jul 18 11:14:02.442129239 (602h3m2.325542862s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442138594 (602h3m2.325552237s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442226743 (602h3m2.325640435s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442714959 (602h3m2.326128728s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442758980 (602h3m2.326172617s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.443182422 (602h3m2.32659606s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.444366357 (602h3m2.327780009s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445017131 (602h3m2.328430777s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445631564 (602h3m2.329045207s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450384259 (602h3m2.333797901s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450888992 (602h3m2.334302623s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451286117 (602h3m2.334699756s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451731093 (602h3m2.33514473s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.452810956 (602h3m2.336224591s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.453311896 (602h3m2.336725535s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:03.061350628 (602h3m2.944764264s) [upload=481367]: Next returned nil segment Jul 18 11:14:03.061375887 (602h3m2.944789525s) [upload=481367]: Waiting for error group managing segments... Jul 18 11:14:03.200779365 (602h3m3.084192987s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.237279929 (602h3m3.120693557s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.238276497 (602h3m3.121690114s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.247803493 (602h3m3.131217125s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.249377712 (602h3m3.132791354s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.253004867 (602h3m3.136418496s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.261597239 (602h3m3.145010874s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.262347910 (602h3m3.145761594s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.291695089 (602h3m3.175108725s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:04.910060448 (602h3m4.793474069s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:32.042078624 (602h3m31.925492241s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042148842 (602h3m31.925562458s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042190707 (602h3m31.925604323s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042239456 (602h3m31.925653071s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042286864 (602h3m31.92570048s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042338367 (602h3m31.925751984s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:37634->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042355181 (602h3m31.925768813s) [seg_pos={0 0}, upload=481367]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 11:14:32.155211809 (602h3m32.038625429s) [upload=481367]: Done uploading segments... Jul 18 11:14:32.155246584 (602h3m32.038660215s) [upload=481367]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.354802873s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.994978089s FAIL (2025-07-18 11:22:50 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.99500434s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 1.049418404s FAIL (2025-07-18 11:22:45 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.049442926s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................X.....X.X..........X............] duration: 2.41711417s FAIL (2025-07-18 11:22:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== g for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:13:13.115856263 (602h2m12.999269893s) [upload=481356]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:13:13.115862143 (602h2m12.999275774s) [upload=481356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:13:13.309475952 (602h2m13.192889568s) [upload=481356]: Done uploading segments... Jul 18 11:13:13.309494751 (602h2m13.192908371s) [upload=481356]: Upload finished. err: <nil> Jul 18 11:14:02.324953677 (602h3m2.208367292s) [upload=481367]: Starting upload Jul 18 11:14:02.325059357 (602h3m2.208472991s) [upload=481367]: Done starting upload Jul 18 11:14:02.325168053 (602h3m2.208581686s) [upload=481367]: Uploading segments... Jul 18 11:14:02.325655202 (602h3m2.20906884s) [upload=481367]: Got next segment. Inline: false Jul 18 11:14:02.325752146 (602h3m2.20916578s) [upload=481367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:14:02.325761218 (602h3m2.209174851s) [upload=481367]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:14:02.441249374 (602h3m2.324663009s) [seg_pos={0 0}, upload=481367]: Begin upload segment... Jul 18 11:14:02.441362061 (602h3m2.324775708s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442051381 (602h3m2.325465045s) [seg_pos={0 0}, upload=481367]: Done begin upload segment. Jul 18 11:14:02.442129239 (602h3m2.325542862s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442138594 (602h3m2.325552237s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442226743 (602h3m2.325640435s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442714959 (602h3m2.326128728s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442758980 (602h3m2.326172617s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.443182422 (602h3m2.32659606s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.444366357 (602h3m2.327780009s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445017131 (602h3m2.328430777s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445631564 (602h3m2.329045207s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450384259 (602h3m2.333797901s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450888992 (602h3m2.334302623s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451286117 (602h3m2.334699756s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451731093 (602h3m2.33514473s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.452810956 (602h3m2.336224591s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.453311896 (602h3m2.336725535s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:03.061350628 (602h3m2.944764264s) [upload=481367]: Next returned nil segment Jul 18 11:14:03.061375887 (602h3m2.944789525s) [upload=481367]: Waiting for error group managing segments... Jul 18 11:14:03.200779365 (602h3m3.084192987s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.237279929 (602h3m3.120693557s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.238276497 (602h3m3.121690114s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.247803493 (602h3m3.131217125s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.249377712 (602h3m3.132791354s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.253004867 (602h3m3.136418496s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.261597239 (602h3m3.145010874s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.262347910 (602h3m3.145761594s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.291695089 (602h3m3.175108725s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:04.910060448 (602h3m4.793474069s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:32.042078624 (602h3m31.925492241s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042148842 (602h3m31.925562458s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042190707 (602h3m31.925604323s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042239456 (602h3m31.925653071s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042286864 (602h3m31.92570048s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042338367 (602h3m31.925751984s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:37634->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042355181 (602h3m31.925768813s) [seg_pos={0 0}, upload=481367]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 11:14:32.155211809 (602h3m32.038625429s) [upload=481367]: Done uploading segments... Jul 18 11:14:32.155246584 (602h3m32.038660215s) [upload=481367]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.417152021s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................X.....X.X..........X...........] duration: 2.628857761s FAIL (2025-07-18 11:22:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== g for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:13:13.115856263 (602h2m12.999269893s) [upload=481356]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:13:13.115862143 (602h2m12.999275774s) [upload=481356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:13:13.309475952 (602h2m13.192889568s) [upload=481356]: Done uploading segments... Jul 18 11:13:13.309494751 (602h2m13.192908371s) [upload=481356]: Upload finished. err: <nil> Jul 18 11:14:02.324953677 (602h3m2.208367292s) [upload=481367]: Starting upload Jul 18 11:14:02.325059357 (602h3m2.208472991s) [upload=481367]: Done starting upload Jul 18 11:14:02.325168053 (602h3m2.208581686s) [upload=481367]: Uploading segments... Jul 18 11:14:02.325655202 (602h3m2.20906884s) [upload=481367]: Got next segment. Inline: false Jul 18 11:14:02.325752146 (602h3m2.20916578s) [upload=481367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:14:02.325761218 (602h3m2.209174851s) [upload=481367]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:14:02.441249374 (602h3m2.324663009s) [seg_pos={0 0}, upload=481367]: Begin upload segment... Jul 18 11:14:02.441362061 (602h3m2.324775708s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442051381 (602h3m2.325465045s) [seg_pos={0 0}, upload=481367]: Done begin upload segment. Jul 18 11:14:02.442129239 (602h3m2.325542862s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442138594 (602h3m2.325552237s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442226743 (602h3m2.325640435s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442714959 (602h3m2.326128728s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442758980 (602h3m2.326172617s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.443182422 (602h3m2.32659606s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.444366357 (602h3m2.327780009s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445017131 (602h3m2.328430777s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445631564 (602h3m2.329045207s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450384259 (602h3m2.333797901s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450888992 (602h3m2.334302623s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451286117 (602h3m2.334699756s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451731093 (602h3m2.33514473s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.452810956 (602h3m2.336224591s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.453311896 (602h3m2.336725535s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:03.061350628 (602h3m2.944764264s) [upload=481367]: Next returned nil segment Jul 18 11:14:03.061375887 (602h3m2.944789525s) [upload=481367]: Waiting for error group managing segments... Jul 18 11:14:03.200779365 (602h3m3.084192987s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.237279929 (602h3m3.120693557s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.238276497 (602h3m3.121690114s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.247803493 (602h3m3.131217125s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.249377712 (602h3m3.132791354s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.253004867 (602h3m3.136418496s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.261597239 (602h3m3.145010874s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.262347910 (602h3m3.145761594s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.291695089 (602h3m3.175108725s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:04.910060448 (602h3m4.793474069s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:32.042078624 (602h3m31.925492241s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042148842 (602h3m31.925562458s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042190707 (602h3m31.925604323s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042239456 (602h3m31.925653071s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042286864 (602h3m31.92570048s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042338367 (602h3m31.925751984s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:37634->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042355181 (602h3m31.925768813s) [seg_pos={0 0}, upload=481367]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 11:14:32.155211809 (602h3m32.038625429s) [upload=481367]: Done uploading segments... Jul 18 11:14:32.155246584 (602h3m32.038660215s) [upload=481367]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.628894454s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 1.104047439s FAIL (2025-07-18 11:22:01 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.104105393s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................X.....X.X..........X..........] duration: 2.482325527s FAIL (2025-07-18 11:21:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== g for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:13:13.115856263 (602h2m12.999269893s) [upload=481356]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:13:13.115862143 (602h2m12.999275774s) [upload=481356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:13:13.309475952 (602h2m13.192889568s) [upload=481356]: Done uploading segments... Jul 18 11:13:13.309494751 (602h2m13.192908371s) [upload=481356]: Upload finished. err: <nil> Jul 18 11:14:02.324953677 (602h3m2.208367292s) [upload=481367]: Starting upload Jul 18 11:14:02.325059357 (602h3m2.208472991s) [upload=481367]: Done starting upload Jul 18 11:14:02.325168053 (602h3m2.208581686s) [upload=481367]: Uploading segments... Jul 18 11:14:02.325655202 (602h3m2.20906884s) [upload=481367]: Got next segment. Inline: false Jul 18 11:14:02.325752146 (602h3m2.20916578s) [upload=481367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:14:02.325761218 (602h3m2.209174851s) [upload=481367]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:14:02.441249374 (602h3m2.324663009s) [seg_pos={0 0}, upload=481367]: Begin upload segment... Jul 18 11:14:02.441362061 (602h3m2.324775708s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442051381 (602h3m2.325465045s) [seg_pos={0 0}, upload=481367]: Done begin upload segment. Jul 18 11:14:02.442129239 (602h3m2.325542862s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442138594 (602h3m2.325552237s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442226743 (602h3m2.325640435s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442714959 (602h3m2.326128728s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442758980 (602h3m2.326172617s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.443182422 (602h3m2.32659606s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.444366357 (602h3m2.327780009s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445017131 (602h3m2.328430777s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445631564 (602h3m2.329045207s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450384259 (602h3m2.333797901s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450888992 (602h3m2.334302623s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451286117 (602h3m2.334699756s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451731093 (602h3m2.33514473s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.452810956 (602h3m2.336224591s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.453311896 (602h3m2.336725535s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:03.061350628 (602h3m2.944764264s) [upload=481367]: Next returned nil segment Jul 18 11:14:03.061375887 (602h3m2.944789525s) [upload=481367]: Waiting for error group managing segments... Jul 18 11:14:03.200779365 (602h3m3.084192987s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.237279929 (602h3m3.120693557s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.238276497 (602h3m3.121690114s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.247803493 (602h3m3.131217125s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.249377712 (602h3m3.132791354s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.253004867 (602h3m3.136418496s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.261597239 (602h3m3.145010874s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.262347910 (602h3m3.145761594s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.291695089 (602h3m3.175108725s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:04.910060448 (602h3m4.793474069s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:32.042078624 (602h3m31.925492241s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042148842 (602h3m31.925562458s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042190707 (602h3m31.925604323s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042239456 (602h3m31.925653071s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042286864 (602h3m31.92570048s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042338367 (602h3m31.925751984s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:37634->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042355181 (602h3m31.925768813s) [seg_pos={0 0}, upload=481367]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 11:14:32.155211809 (602h3m32.038625429s) [upload=481367]: Done uploading segments... Jul 18 11:14:32.155246584 (602h3m32.038660215s) [upload=481367]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.482350965s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.611328887s FAIL (2025-07-18 11:21:38 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.611360603s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 1.019120747s FAIL (2025-07-18 11:21:09 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.019143929s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.71889094s FAIL (2025-07-18 11:21:02 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.718923658s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................X.....X.X..........X.........] duration: 2.570110941s FAIL (2025-07-18 11:20:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== g for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:13:13.115856263 (602h2m12.999269893s) [upload=481356]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:13:13.115862143 (602h2m12.999275774s) [upload=481356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:13:13.309475952 (602h2m13.192889568s) [upload=481356]: Done uploading segments... Jul 18 11:13:13.309494751 (602h2m13.192908371s) [upload=481356]: Upload finished. err: <nil> Jul 18 11:14:02.324953677 (602h3m2.208367292s) [upload=481367]: Starting upload Jul 18 11:14:02.325059357 (602h3m2.208472991s) [upload=481367]: Done starting upload Jul 18 11:14:02.325168053 (602h3m2.208581686s) [upload=481367]: Uploading segments... Jul 18 11:14:02.325655202 (602h3m2.20906884s) [upload=481367]: Got next segment. Inline: false Jul 18 11:14:02.325752146 (602h3m2.20916578s) [upload=481367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:14:02.325761218 (602h3m2.209174851s) [upload=481367]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:14:02.441249374 (602h3m2.324663009s) [seg_pos={0 0}, upload=481367]: Begin upload segment... Jul 18 11:14:02.441362061 (602h3m2.324775708s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442051381 (602h3m2.325465045s) [seg_pos={0 0}, upload=481367]: Done begin upload segment. Jul 18 11:14:02.442129239 (602h3m2.325542862s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442138594 (602h3m2.325552237s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442226743 (602h3m2.325640435s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442714959 (602h3m2.326128728s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442758980 (602h3m2.326172617s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.443182422 (602h3m2.32659606s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.444366357 (602h3m2.327780009s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445017131 (602h3m2.328430777s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445631564 (602h3m2.329045207s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450384259 (602h3m2.333797901s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450888992 (602h3m2.334302623s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451286117 (602h3m2.334699756s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451731093 (602h3m2.33514473s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.452810956 (602h3m2.336224591s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.453311896 (602h3m2.336725535s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:03.061350628 (602h3m2.944764264s) [upload=481367]: Next returned nil segment Jul 18 11:14:03.061375887 (602h3m2.944789525s) [upload=481367]: Waiting for error group managing segments... Jul 18 11:14:03.200779365 (602h3m3.084192987s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.237279929 (602h3m3.120693557s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.238276497 (602h3m3.121690114s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.247803493 (602h3m3.131217125s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.249377712 (602h3m3.132791354s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.253004867 (602h3m3.136418496s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.261597239 (602h3m3.145010874s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.262347910 (602h3m3.145761594s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.291695089 (602h3m3.175108725s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:04.910060448 (602h3m4.793474069s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:32.042078624 (602h3m31.925492241s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042148842 (602h3m31.925562458s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042190707 (602h3m31.925604323s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042239456 (602h3m31.925653071s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042286864 (602h3m31.92570048s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042338367 (602h3m31.925751984s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:37634->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042355181 (602h3m31.925768813s) [seg_pos={0 0}, upload=481367]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 11:14:32.155211809 (602h3m32.038625429s) [upload=481367]: Done uploading segments... Jul 18 11:14:32.155246584 (602h3m32.038660215s) [upload=481367]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.570219162s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 3.059947554s FAIL (2025-07-18 11:20:30 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 3.059976135s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 1.506143399s FAIL (2025-07-18 11:20:07 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.506202453s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................X.....X.X..........X........] duration: 2.793987875s FAIL (2025-07-18 11:20:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== g for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:13:13.115856263 (602h2m12.999269893s) [upload=481356]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:13:13.115862143 (602h2m12.999275774s) [upload=481356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:13:13.309475952 (602h2m13.192889568s) [upload=481356]: Done uploading segments... Jul 18 11:13:13.309494751 (602h2m13.192908371s) [upload=481356]: Upload finished. err: <nil> Jul 18 11:14:02.324953677 (602h3m2.208367292s) [upload=481367]: Starting upload Jul 18 11:14:02.325059357 (602h3m2.208472991s) [upload=481367]: Done starting upload Jul 18 11:14:02.325168053 (602h3m2.208581686s) [upload=481367]: Uploading segments... Jul 18 11:14:02.325655202 (602h3m2.20906884s) [upload=481367]: Got next segment. Inline: false Jul 18 11:14:02.325752146 (602h3m2.20916578s) [upload=481367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:14:02.325761218 (602h3m2.209174851s) [upload=481367]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:14:02.441249374 (602h3m2.324663009s) [seg_pos={0 0}, upload=481367]: Begin upload segment... Jul 18 11:14:02.441362061 (602h3m2.324775708s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442051381 (602h3m2.325465045s) [seg_pos={0 0}, upload=481367]: Done begin upload segment. Jul 18 11:14:02.442129239 (602h3m2.325542862s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442138594 (602h3m2.325552237s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442226743 (602h3m2.325640435s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442714959 (602h3m2.326128728s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442758980 (602h3m2.326172617s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.443182422 (602h3m2.32659606s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.444366357 (602h3m2.327780009s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445017131 (602h3m2.328430777s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445631564 (602h3m2.329045207s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450384259 (602h3m2.333797901s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450888992 (602h3m2.334302623s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451286117 (602h3m2.334699756s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451731093 (602h3m2.33514473s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.452810956 (602h3m2.336224591s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.453311896 (602h3m2.336725535s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:03.061350628 (602h3m2.944764264s) [upload=481367]: Next returned nil segment Jul 18 11:14:03.061375887 (602h3m2.944789525s) [upload=481367]: Waiting for error group managing segments... Jul 18 11:14:03.200779365 (602h3m3.084192987s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.237279929 (602h3m3.120693557s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.238276497 (602h3m3.121690114s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.247803493 (602h3m3.131217125s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.249377712 (602h3m3.132791354s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.253004867 (602h3m3.136418496s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.261597239 (602h3m3.145010874s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.262347910 (602h3m3.145761594s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.291695089 (602h3m3.175108725s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:04.910060448 (602h3m4.793474069s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:32.042078624 (602h3m31.925492241s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042148842 (602h3m31.925562458s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042190707 (602h3m31.925604323s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042239456 (602h3m31.925653071s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042286864 (602h3m31.92570048s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042338367 (602h3m31.925751984s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:37634->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042355181 (602h3m31.925768813s) [seg_pos={0 0}, upload=481367]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 11:14:32.155211809 (602h3m32.038625429s) [upload=481367]: Done uploading segments... Jul 18 11:14:32.155246584 (602h3m32.038660215s) [upload=481367]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.794076576s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 3.565546099s FAIL (2025-07-18 11:19:40 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 3.56557148s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 1.640634056s FAIL (2025-07-18 11:19:40 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.640934428s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 1.504574275s FAIL (2025-07-18 11:19:09 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.504590678s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................X.....X.X..........X.......] duration: 3.542814092s FAIL (2025-07-18 11:18:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== g for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:13:13.115856263 (602h2m12.999269893s) [upload=481356]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:13:13.115862143 (602h2m12.999275774s) [upload=481356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:13:13.309475952 (602h2m13.192889568s) [upload=481356]: Done uploading segments... Jul 18 11:13:13.309494751 (602h2m13.192908371s) [upload=481356]: Upload finished. err: <nil> Jul 18 11:14:02.324953677 (602h3m2.208367292s) [upload=481367]: Starting upload Jul 18 11:14:02.325059357 (602h3m2.208472991s) [upload=481367]: Done starting upload Jul 18 11:14:02.325168053 (602h3m2.208581686s) [upload=481367]: Uploading segments... Jul 18 11:14:02.325655202 (602h3m2.20906884s) [upload=481367]: Got next segment. Inline: false Jul 18 11:14:02.325752146 (602h3m2.20916578s) [upload=481367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:14:02.325761218 (602h3m2.209174851s) [upload=481367]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:14:02.441249374 (602h3m2.324663009s) [seg_pos={0 0}, upload=481367]: Begin upload segment... Jul 18 11:14:02.441362061 (602h3m2.324775708s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442051381 (602h3m2.325465045s) [seg_pos={0 0}, upload=481367]: Done begin upload segment. Jul 18 11:14:02.442129239 (602h3m2.325542862s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442138594 (602h3m2.325552237s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442226743 (602h3m2.325640435s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442714959 (602h3m2.326128728s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442758980 (602h3m2.326172617s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.443182422 (602h3m2.32659606s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.444366357 (602h3m2.327780009s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445017131 (602h3m2.328430777s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445631564 (602h3m2.329045207s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450384259 (602h3m2.333797901s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450888992 (602h3m2.334302623s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451286117 (602h3m2.334699756s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451731093 (602h3m2.33514473s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.452810956 (602h3m2.336224591s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.453311896 (602h3m2.336725535s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:03.061350628 (602h3m2.944764264s) [upload=481367]: Next returned nil segment Jul 18 11:14:03.061375887 (602h3m2.944789525s) [upload=481367]: Waiting for error group managing segments... Jul 18 11:14:03.200779365 (602h3m3.084192987s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.237279929 (602h3m3.120693557s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.238276497 (602h3m3.121690114s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.247803493 (602h3m3.131217125s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.249377712 (602h3m3.132791354s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.253004867 (602h3m3.136418496s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.261597239 (602h3m3.145010874s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.262347910 (602h3m3.145761594s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.291695089 (602h3m3.175108725s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:04.910060448 (602h3m4.793474069s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:32.042078624 (602h3m31.925492241s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042148842 (602h3m31.925562458s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042190707 (602h3m31.925604323s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042239456 (602h3m31.925653071s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042286864 (602h3m31.92570048s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042338367 (602h3m31.925751984s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:37634->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042355181 (602h3m31.925768813s) [seg_pos={0 0}, upload=481367]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 11:14:32.155211809 (602h3m32.038625429s) [upload=481367]: Done uploading segments... Jul 18 11:14:32.155246584 (602h3m32.038660215s) [upload=481367]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 3.542843035s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.594696192s FAIL (2025-07-18 11:18:37 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.594730074s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................X.....X.X..........X......] duration: 2.804230765s FAIL (2025-07-18 11:18:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== g for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:13:13.115856263 (602h2m12.999269893s) [upload=481356]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:13:13.115862143 (602h2m12.999275774s) [upload=481356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:13:13.309475952 (602h2m13.192889568s) [upload=481356]: Done uploading segments... Jul 18 11:13:13.309494751 (602h2m13.192908371s) [upload=481356]: Upload finished. err: <nil> Jul 18 11:14:02.324953677 (602h3m2.208367292s) [upload=481367]: Starting upload Jul 18 11:14:02.325059357 (602h3m2.208472991s) [upload=481367]: Done starting upload Jul 18 11:14:02.325168053 (602h3m2.208581686s) [upload=481367]: Uploading segments... Jul 18 11:14:02.325655202 (602h3m2.20906884s) [upload=481367]: Got next segment. Inline: false Jul 18 11:14:02.325752146 (602h3m2.20916578s) [upload=481367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:14:02.325761218 (602h3m2.209174851s) [upload=481367]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:14:02.441249374 (602h3m2.324663009s) [seg_pos={0 0}, upload=481367]: Begin upload segment... Jul 18 11:14:02.441362061 (602h3m2.324775708s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442051381 (602h3m2.325465045s) [seg_pos={0 0}, upload=481367]: Done begin upload segment. Jul 18 11:14:02.442129239 (602h3m2.325542862s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442138594 (602h3m2.325552237s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442226743 (602h3m2.325640435s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442714959 (602h3m2.326128728s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442758980 (602h3m2.326172617s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.443182422 (602h3m2.32659606s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.444366357 (602h3m2.327780009s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445017131 (602h3m2.328430777s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445631564 (602h3m2.329045207s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450384259 (602h3m2.333797901s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450888992 (602h3m2.334302623s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451286117 (602h3m2.334699756s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451731093 (602h3m2.33514473s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.452810956 (602h3m2.336224591s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.453311896 (602h3m2.336725535s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:03.061350628 (602h3m2.944764264s) [upload=481367]: Next returned nil segment Jul 18 11:14:03.061375887 (602h3m2.944789525s) [upload=481367]: Waiting for error group managing segments... Jul 18 11:14:03.200779365 (602h3m3.084192987s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.237279929 (602h3m3.120693557s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.238276497 (602h3m3.121690114s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.247803493 (602h3m3.131217125s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.249377712 (602h3m3.132791354s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.253004867 (602h3m3.136418496s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.261597239 (602h3m3.145010874s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.262347910 (602h3m3.145761594s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.291695089 (602h3m3.175108725s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:04.910060448 (602h3m4.793474069s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:32.042078624 (602h3m31.925492241s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042148842 (602h3m31.925562458s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042190707 (602h3m31.925604323s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042239456 (602h3m31.925653071s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042286864 (602h3m31.92570048s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042338367 (602h3m31.925751984s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:37634->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042355181 (602h3m31.925768813s) [seg_pos={0 0}, upload=481367]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 11:14:32.155211809 (602h3m32.038625429s) [upload=481367]: Done uploading segments... Jul 18 11:14:32.155246584 (602h3m32.038660215s) [upload=481367]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.804318491s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 1.197559671s FAIL (2025-07-18 11:18:01 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.197607598s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.742903761s FAIL (2025-07-18 11:17:55 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.742943417s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 1.121825819s FAIL (2025-07-18 11:17:29 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.121855802s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................X.....X.X..........X.....] duration: 2.444781006s FAIL (2025-07-18 11:17:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== g for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:13:13.115856263 (602h2m12.999269893s) [upload=481356]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:13:13.115862143 (602h2m12.999275774s) [upload=481356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:13:13.309475952 (602h2m13.192889568s) [upload=481356]: Done uploading segments... Jul 18 11:13:13.309494751 (602h2m13.192908371s) [upload=481356]: Upload finished. err: <nil> Jul 18 11:14:02.324953677 (602h3m2.208367292s) [upload=481367]: Starting upload Jul 18 11:14:02.325059357 (602h3m2.208472991s) [upload=481367]: Done starting upload Jul 18 11:14:02.325168053 (602h3m2.208581686s) [upload=481367]: Uploading segments... Jul 18 11:14:02.325655202 (602h3m2.20906884s) [upload=481367]: Got next segment. Inline: false Jul 18 11:14:02.325752146 (602h3m2.20916578s) [upload=481367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:14:02.325761218 (602h3m2.209174851s) [upload=481367]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:14:02.441249374 (602h3m2.324663009s) [seg_pos={0 0}, upload=481367]: Begin upload segment... Jul 18 11:14:02.441362061 (602h3m2.324775708s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442051381 (602h3m2.325465045s) [seg_pos={0 0}, upload=481367]: Done begin upload segment. Jul 18 11:14:02.442129239 (602h3m2.325542862s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442138594 (602h3m2.325552237s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442226743 (602h3m2.325640435s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442714959 (602h3m2.326128728s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442758980 (602h3m2.326172617s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.443182422 (602h3m2.32659606s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.444366357 (602h3m2.327780009s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445017131 (602h3m2.328430777s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445631564 (602h3m2.329045207s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450384259 (602h3m2.333797901s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450888992 (602h3m2.334302623s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451286117 (602h3m2.334699756s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451731093 (602h3m2.33514473s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.452810956 (602h3m2.336224591s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.453311896 (602h3m2.336725535s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:03.061350628 (602h3m2.944764264s) [upload=481367]: Next returned nil segment Jul 18 11:14:03.061375887 (602h3m2.944789525s) [upload=481367]: Waiting for error group managing segments... Jul 18 11:14:03.200779365 (602h3m3.084192987s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.237279929 (602h3m3.120693557s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.238276497 (602h3m3.121690114s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.247803493 (602h3m3.131217125s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.249377712 (602h3m3.132791354s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.253004867 (602h3m3.136418496s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.261597239 (602h3m3.145010874s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.262347910 (602h3m3.145761594s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.291695089 (602h3m3.175108725s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:04.910060448 (602h3m4.793474069s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:32.042078624 (602h3m31.925492241s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042148842 (602h3m31.925562458s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042190707 (602h3m31.925604323s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042239456 (602h3m31.925653071s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042286864 (602h3m31.92570048s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042338367 (602h3m31.925751984s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:37634->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042355181 (602h3m31.925768813s) [seg_pos={0 0}, upload=481367]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 11:14:32.155211809 (602h3m32.038625429s) [upload=481367]: Done uploading segments... Jul 18 11:14:32.155246584 (602h3m32.038660215s) [upload=481367]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.444811718s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.706160077s FAIL (2025-07-18 11:17:08 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.706207319s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.628550048s FAIL (2025-07-18 11:16:45 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.628584342s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................................X.....X.X..........X....] duration: 2.412284274s FAIL (2025-07-18 11:16:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== g for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:13:13.115856263 (602h2m12.999269893s) [upload=481356]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:13:13.115862143 (602h2m12.999275774s) [upload=481356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:13:13.309475952 (602h2m13.192889568s) [upload=481356]: Done uploading segments... Jul 18 11:13:13.309494751 (602h2m13.192908371s) [upload=481356]: Upload finished. err: <nil> Jul 18 11:14:02.324953677 (602h3m2.208367292s) [upload=481367]: Starting upload Jul 18 11:14:02.325059357 (602h3m2.208472991s) [upload=481367]: Done starting upload Jul 18 11:14:02.325168053 (602h3m2.208581686s) [upload=481367]: Uploading segments... Jul 18 11:14:02.325655202 (602h3m2.20906884s) [upload=481367]: Got next segment. Inline: false Jul 18 11:14:02.325752146 (602h3m2.20916578s) [upload=481367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:14:02.325761218 (602h3m2.209174851s) [upload=481367]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:14:02.441249374 (602h3m2.324663009s) [seg_pos={0 0}, upload=481367]: Begin upload segment... Jul 18 11:14:02.441362061 (602h3m2.324775708s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442051381 (602h3m2.325465045s) [seg_pos={0 0}, upload=481367]: Done begin upload segment. Jul 18 11:14:02.442129239 (602h3m2.325542862s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442138594 (602h3m2.325552237s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442226743 (602h3m2.325640435s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442714959 (602h3m2.326128728s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442758980 (602h3m2.326172617s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.443182422 (602h3m2.32659606s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.444366357 (602h3m2.327780009s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445017131 (602h3m2.328430777s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445631564 (602h3m2.329045207s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450384259 (602h3m2.333797901s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450888992 (602h3m2.334302623s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451286117 (602h3m2.334699756s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451731093 (602h3m2.33514473s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.452810956 (602h3m2.336224591s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.453311896 (602h3m2.336725535s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:03.061350628 (602h3m2.944764264s) [upload=481367]: Next returned nil segment Jul 18 11:14:03.061375887 (602h3m2.944789525s) [upload=481367]: Waiting for error group managing segments... Jul 18 11:14:03.200779365 (602h3m3.084192987s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.237279929 (602h3m3.120693557s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.238276497 (602h3m3.121690114s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.247803493 (602h3m3.131217125s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.249377712 (602h3m3.132791354s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.253004867 (602h3m3.136418496s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.261597239 (602h3m3.145010874s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.262347910 (602h3m3.145761594s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.291695089 (602h3m3.175108725s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:04.910060448 (602h3m4.793474069s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:32.042078624 (602h3m31.925492241s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042148842 (602h3m31.925562458s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042190707 (602h3m31.925604323s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042239456 (602h3m31.925653071s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042286864 (602h3m31.92570048s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042338367 (602h3m31.925751984s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:37634->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042355181 (602h3m31.925768813s) [seg_pos={0 0}, upload=481367]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 11:14:32.155211809 (602h3m32.038625429s) [upload=481367]: Done uploading segments... Jul 18 11:14:32.155246584 (602h3m32.038660215s) [upload=481367]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.412317238s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 1.106959216s FAIL (2025-07-18 11:16:21 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.10698411s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................................X.....X.X..........X...] duration: 2.336349388s FAIL (2025-07-18 11:16:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== g for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:13:13.115856263 (602h2m12.999269893s) [upload=481356]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:13:13.115862143 (602h2m12.999275774s) [upload=481356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:13:13.309475952 (602h2m13.192889568s) [upload=481356]: Done uploading segments... Jul 18 11:13:13.309494751 (602h2m13.192908371s) [upload=481356]: Upload finished. err: <nil> Jul 18 11:14:02.324953677 (602h3m2.208367292s) [upload=481367]: Starting upload Jul 18 11:14:02.325059357 (602h3m2.208472991s) [upload=481367]: Done starting upload Jul 18 11:14:02.325168053 (602h3m2.208581686s) [upload=481367]: Uploading segments... Jul 18 11:14:02.325655202 (602h3m2.20906884s) [upload=481367]: Got next segment. Inline: false Jul 18 11:14:02.325752146 (602h3m2.20916578s) [upload=481367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:14:02.325761218 (602h3m2.209174851s) [upload=481367]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:14:02.441249374 (602h3m2.324663009s) [seg_pos={0 0}, upload=481367]: Begin upload segment... Jul 18 11:14:02.441362061 (602h3m2.324775708s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442051381 (602h3m2.325465045s) [seg_pos={0 0}, upload=481367]: Done begin upload segment. Jul 18 11:14:02.442129239 (602h3m2.325542862s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442138594 (602h3m2.325552237s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442226743 (602h3m2.325640435s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442714959 (602h3m2.326128728s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442758980 (602h3m2.326172617s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.443182422 (602h3m2.32659606s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.444366357 (602h3m2.327780009s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445017131 (602h3m2.328430777s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445631564 (602h3m2.329045207s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450384259 (602h3m2.333797901s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450888992 (602h3m2.334302623s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451286117 (602h3m2.334699756s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451731093 (602h3m2.33514473s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.452810956 (602h3m2.336224591s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.453311896 (602h3m2.336725535s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:03.061350628 (602h3m2.944764264s) [upload=481367]: Next returned nil segment Jul 18 11:14:03.061375887 (602h3m2.944789525s) [upload=481367]: Waiting for error group managing segments... Jul 18 11:14:03.200779365 (602h3m3.084192987s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.237279929 (602h3m3.120693557s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.238276497 (602h3m3.121690114s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.247803493 (602h3m3.131217125s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.249377712 (602h3m3.132791354s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.253004867 (602h3m3.136418496s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.261597239 (602h3m3.145010874s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.262347910 (602h3m3.145761594s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.291695089 (602h3m3.175108725s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:04.910060448 (602h3m4.793474069s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:32.042078624 (602h3m31.925492241s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042148842 (602h3m31.925562458s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042190707 (602h3m31.925604323s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042239456 (602h3m31.925653071s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042286864 (602h3m31.92570048s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042338367 (602h3m31.925751984s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:37634->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042355181 (602h3m31.925768813s) [seg_pos={0 0}, upload=481367]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 11:14:32.155211809 (602h3m32.038625429s) [upload=481367]: Done uploading segments... Jul 18 11:14:32.155246584 (602h3m32.038660215s) [upload=481367]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.336379564s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 1.004900467s FAIL (2025-07-18 11:15:41 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.004936509s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.723567388s FAIL (2025-07-18 11:15:37 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.723597855s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................................X.....X.X..........X..] duration: 2.347295959s FAIL (2025-07-18 11:15:16 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== g for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:13:13.115856263 (602h2m12.999269893s) [upload=481356]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:13:13.115862143 (602h2m12.999275774s) [upload=481356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:13:13.309475952 (602h2m13.192889568s) [upload=481356]: Done uploading segments... Jul 18 11:13:13.309494751 (602h2m13.192908371s) [upload=481356]: Upload finished. err: <nil> Jul 18 11:14:02.324953677 (602h3m2.208367292s) [upload=481367]: Starting upload Jul 18 11:14:02.325059357 (602h3m2.208472991s) [upload=481367]: Done starting upload Jul 18 11:14:02.325168053 (602h3m2.208581686s) [upload=481367]: Uploading segments... Jul 18 11:14:02.325655202 (602h3m2.20906884s) [upload=481367]: Got next segment. Inline: false Jul 18 11:14:02.325752146 (602h3m2.20916578s) [upload=481367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:14:02.325761218 (602h3m2.209174851s) [upload=481367]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:14:02.441249374 (602h3m2.324663009s) [seg_pos={0 0}, upload=481367]: Begin upload segment... Jul 18 11:14:02.441362061 (602h3m2.324775708s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442051381 (602h3m2.325465045s) [seg_pos={0 0}, upload=481367]: Done begin upload segment. Jul 18 11:14:02.442129239 (602h3m2.325542862s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442138594 (602h3m2.325552237s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442226743 (602h3m2.325640435s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442714959 (602h3m2.326128728s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442758980 (602h3m2.326172617s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.443182422 (602h3m2.32659606s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.444366357 (602h3m2.327780009s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445017131 (602h3m2.328430777s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445631564 (602h3m2.329045207s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450384259 (602h3m2.333797901s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450888992 (602h3m2.334302623s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451286117 (602h3m2.334699756s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451731093 (602h3m2.33514473s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.452810956 (602h3m2.336224591s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.453311896 (602h3m2.336725535s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:03.061350628 (602h3m2.944764264s) [upload=481367]: Next returned nil segment Jul 18 11:14:03.061375887 (602h3m2.944789525s) [upload=481367]: Waiting for error group managing segments... Jul 18 11:14:03.200779365 (602h3m3.084192987s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.237279929 (602h3m3.120693557s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.238276497 (602h3m3.121690114s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.247803493 (602h3m3.131217125s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.249377712 (602h3m3.132791354s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.253004867 (602h3m3.136418496s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.261597239 (602h3m3.145010874s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.262347910 (602h3m3.145761594s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.291695089 (602h3m3.175108725s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:04.910060448 (602h3m4.793474069s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:32.042078624 (602h3m31.925492241s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042148842 (602h3m31.925562458s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042190707 (602h3m31.925604323s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042239456 (602h3m31.925653071s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042286864 (602h3m31.92570048s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042338367 (602h3m31.925751984s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:37634->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042355181 (602h3m31.925768813s) [seg_pos={0 0}, upload=481367]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 11:14:32.155211809 (602h3m32.038625429s) [upload=481367]: Done uploading segments... Jul 18 11:14:32.155246584 (602h3m32.038660215s) [upload=481367]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.347391128s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.612584989s FAIL (2025-07-18 11:14:54 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.612637853s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................X.....X.X..........X.] duration: 2.294308913s FAIL (2025-07-18 11:14:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== g for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:13:13.115856263 (602h2m12.999269893s) [upload=481356]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:13:13.115862143 (602h2m12.999275774s) [upload=481356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:13:13.309475952 (602h2m13.192889568s) [upload=481356]: Done uploading segments... Jul 18 11:13:13.309494751 (602h2m13.192908371s) [upload=481356]: Upload finished. err: <nil> Jul 18 11:14:02.324953677 (602h3m2.208367292s) [upload=481367]: Starting upload Jul 18 11:14:02.325059357 (602h3m2.208472991s) [upload=481367]: Done starting upload Jul 18 11:14:02.325168053 (602h3m2.208581686s) [upload=481367]: Uploading segments... Jul 18 11:14:02.325655202 (602h3m2.20906884s) [upload=481367]: Got next segment. Inline: false Jul 18 11:14:02.325752146 (602h3m2.20916578s) [upload=481367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:14:02.325761218 (602h3m2.209174851s) [upload=481367]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:14:02.441249374 (602h3m2.324663009s) [seg_pos={0 0}, upload=481367]: Begin upload segment... Jul 18 11:14:02.441362061 (602h3m2.324775708s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442051381 (602h3m2.325465045s) [seg_pos={0 0}, upload=481367]: Done begin upload segment. Jul 18 11:14:02.442129239 (602h3m2.325542862s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442138594 (602h3m2.325552237s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442226743 (602h3m2.325640435s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442714959 (602h3m2.326128728s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442758980 (602h3m2.326172617s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.443182422 (602h3m2.32659606s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.444366357 (602h3m2.327780009s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445017131 (602h3m2.328430777s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445631564 (602h3m2.329045207s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450384259 (602h3m2.333797901s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450888992 (602h3m2.334302623s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451286117 (602h3m2.334699756s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451731093 (602h3m2.33514473s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.452810956 (602h3m2.336224591s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.453311896 (602h3m2.336725535s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:03.061350628 (602h3m2.944764264s) [upload=481367]: Next returned nil segment Jul 18 11:14:03.061375887 (602h3m2.944789525s) [upload=481367]: Waiting for error group managing segments... Jul 18 11:14:03.200779365 (602h3m3.084192987s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.237279929 (602h3m3.120693557s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.238276497 (602h3m3.121690114s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.247803493 (602h3m3.131217125s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.249377712 (602h3m3.132791354s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.253004867 (602h3m3.136418496s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.261597239 (602h3m3.145010874s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.262347910 (602h3m3.145761594s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.291695089 (602h3m3.175108725s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:04.910060448 (602h3m4.793474069s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:32.042078624 (602h3m31.925492241s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042148842 (602h3m31.925562458s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042190707 (602h3m31.925604323s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042239456 (602h3m31.925653071s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042286864 (602h3m31.92570048s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042338367 (602h3m31.925751984s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:37634->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042355181 (602h3m31.925768813s) [seg_pos={0 0}, upload=481367]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 11:14:32.155211809 (602h3m32.038625429s) [upload=481367]: Done uploading segments... Jul 18 11:14:32.155246584 (602h3m32.038660215s) [upload=481367]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.294347241s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................X.....X.X..........X] duration: 30.115859923s FAIL (2025-07-18 11:14:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== g for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:13:13.115856263 (602h2m12.999269893s) [upload=481356]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:13:13.115862143 (602h2m12.999275774s) [upload=481356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:13:13.309475952 (602h2m13.192889568s) [upload=481356]: Done uploading segments... Jul 18 11:13:13.309494751 (602h2m13.192908371s) [upload=481356]: Upload finished. err: <nil> Jul 18 11:14:02.324953677 (602h3m2.208367292s) [upload=481367]: Starting upload Jul 18 11:14:02.325059357 (602h3m2.208472991s) [upload=481367]: Done starting upload Jul 18 11:14:02.325168053 (602h3m2.208581686s) [upload=481367]: Uploading segments... Jul 18 11:14:02.325655202 (602h3m2.20906884s) [upload=481367]: Got next segment. Inline: false Jul 18 11:14:02.325752146 (602h3m2.20916578s) [upload=481367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:14:02.325761218 (602h3m2.209174851s) [upload=481367]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:14:02.441249374 (602h3m2.324663009s) [seg_pos={0 0}, upload=481367]: Begin upload segment... Jul 18 11:14:02.441362061 (602h3m2.324775708s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442051381 (602h3m2.325465045s) [seg_pos={0 0}, upload=481367]: Done begin upload segment. Jul 18 11:14:02.442129239 (602h3m2.325542862s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442138594 (602h3m2.325552237s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442226743 (602h3m2.325640435s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442714959 (602h3m2.326128728s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.442758980 (602h3m2.326172617s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.443182422 (602h3m2.32659606s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.444366357 (602h3m2.327780009s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445017131 (602h3m2.328430777s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.445631564 (602h3m2.329045207s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450384259 (602h3m2.333797901s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.450888992 (602h3m2.334302623s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451286117 (602h3m2.334699756s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.451731093 (602h3m2.33514473s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.452810956 (602h3m2.336224591s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:02.453311896 (602h3m2.336725535s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Uploading piece... Jul 18 11:14:03.061350628 (602h3m2.944764264s) [upload=481367]: Next returned nil segment Jul 18 11:14:03.061375887 (602h3m2.944789525s) [upload=481367]: Waiting for error group managing segments... Jul 18 11:14:03.200779365 (602h3m3.084192987s) [noise=true , address=77.90.31.10:4519 , piece_id=QCYKINOLHQOKDPFLH63UFUTKWCOPKR4P4Q6P77HTIKT32HE6I2CA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.237279929 (602h3m3.120693557s) [noise=true , address=77.90.31.19:4842 , piece_id=PTUKG4WCTW2YP74R6ZDCEQTIRYX32WV2JGU6WK2TBEZXIDTTDMLA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.238276497 (602h3m3.121690114s) [noise=true , address=45.91.116.14:4190 , piece_id=GUC5P2CER6O4O5ARNK6W36YRBJVD6CRZZBUAHDVKAG7JI3KLDTFA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.247803493 (602h3m3.131217125s) [noise=true , address=104.36.82.2:4729 , piece_id=5XGFMXLXLVTGSLCMBWSDFSQZERKZHZFAZWDVF4XIKW4FJ2NQ2UOQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.249377712 (602h3m3.132791354s) [noise=true , address=104.36.82.7:4910 , piece_id=UYBHDOQT3PZVBVVRJ4PYQIGEZGJK4MPBU44SXEZLFSZFCH4IFVZQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.253004867 (602h3m3.136418496s) [noise=true , address=104.36.82.11:4115 , piece_id=DET5WP3ZRQUCFLKLKNZJ5GQ5MSRCKZ7YLAHMASRR556B56SGGQBQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.261597239 (602h3m3.145010874s) [noise=true , address=104.36.82.13:4186 , piece_id=EQY7YUPEGVIBJPEK7ZF4I2D7ELRHV4RYOZSUO2QL6H6P57D4ZU2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.262347910 (602h3m3.145761594s) [noise=true , address=104.36.82.13:4185 , piece_id=ZZ4GGQPAZGUX2QZLIOOKKFXUAHCKYEA7RNSTTZOD6EBFEQYPU5PQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:03.291695089 (602h3m3.175108725s) [noise=true , address=104.36.82.3:4765 , piece_id=QZ3USMXV64HCYXL53U2EJMEPCTWR2B7QNSBECIIT2ZQIEJN3C66A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:04.910060448 (602h3m4.793474069s) [noise=true , address=45.91.116.10:4083 , piece_id=I3KXTJ55NYKK3I6CMZVXHZHFIHHCSUXAFGXLR3J357WJFUMGSJ2A, seg_pos={0 0}, upload=481367]: Done uploading piece. err: <nil> Jul 18 11:14:32.042078624 (602h3m31.925492241s) [noise=true , address=77.90.31.21:4878 , piece_id=WLIWPKNFPCA34KMTDZ55HBESMS2QD5Y5M7A5HYV7M7QGWSLPT2MA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042148842 (602h3m31.925562458s) [noise=true , address=77.90.31.17:4771 , piece_id=Q2QOXDUWIECGCCP6JIKCXRF56NRALJ3P5XLUYKYSG3D37KJSN6YQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042190707 (602h3m31.925604323s) [noise=true , address=77.90.31.11:4555 , piece_id=5R3KNXQVPB7CNJU53FF4ENM2MH2737X57UENZJXAD7ASY6EURJPQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042239456 (602h3m31.925653071s) [noise=true , address=77.90.31.18:4806 , piece_id=OYHBXW5PR46UIVHJUOKXI34XGVPZFUQHDZPIUS4X5V254YTL4YCA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042286864 (602h3m31.92570048s) [noise=true , address=77.90.31.17:4770 , piece_id=YR4BZUAGSG3VUG5OI7OZKNQZ6PTWPPAIVTC4TBLPDN7FZLCNC3SQ, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042338367 (602h3m31.925751984s) [noise=true , address=77.90.31.18:4807 , piece_id=7VE3GDNIZIK5CJFOKPGV2PNEYFQOQAKRD7A7QRG3IAA6TZFOKZXA, seg_pos={0 0}, upload=481367]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:37634->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:14:32.042355181 (602h3m31.925768813s) [seg_pos={0 0}, upload=481367]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 11:14:32.155211809 (602h3m32.038625429s) [upload=481367]: Done uploading segments... Jul 18 11:14:32.155246584 (602h3m32.038660215s) [upload=481367]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.115892066s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)