Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 1.085745356s FAIL (2025-07-18 11:39:30 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.085778661s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.668105047s FAIL (2025-07-18 11:39:22 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.668157168s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......X.....X.X..........X...............X..........X.......] duration: 2.362673185s FAIL (2025-07-18 11:38:51 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== , upload=481608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:32:54.154838589 (602h21m54.038252206s) [upload=481608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:32:54.154843826 (602h21m54.038257453s) [upload=481608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:32:54.305725283 (602h21m54.189138898s) [upload=481608]: Done uploading segments... Jul 18 11:32:54.305745379 (602h21m54.189159007s) [upload=481608]: Upload finished. err: <nil> Jul 18 11:33:53.306124309 (602h22m53.189537928s) [upload=481619]: Starting upload Jul 18 11:33:53.306205552 (602h22m53.18961919s) [upload=481619]: Done starting upload Jul 18 11:33:53.306275544 (602h22m53.189689161s) [upload=481619]: Uploading segments... Jul 18 11:33:53.306475959 (602h22m53.189889586s) [upload=481619]: Got next segment. Inline: false Jul 18 11:33:53.306513703 (602h22m53.18992732s) [upload=481619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:33:53.306519968 (602h22m53.189933586s) [upload=481619]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:33:53.419183467 (602h22m53.302597096s) [seg_pos={0 0}, upload=481619]: Begin upload segment... Jul 18 11:33:53.419316113 (602h22m53.302729751s) [noise=true , address=45.91.116.12:4227 , piece_id=YZHWNLTXU4C7U6BD4ZMOASYHRKWOJX4SHTYSZCBFSCUCSIQGWKOA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.419850102 (602h22m53.303263734s) [noise=true , address=77.90.31.15:4699 , piece_id=YJBX34SLFMFNBBFJA3G2KDKB4HGIWY743XTQ5OHVFE5RJUDKYLXA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.420226364 (602h22m53.30364s) [noise=true , address=45.91.116.10:4082 , piece_id=MTW24CAAS7GMDLOJPXNHI6GEAZMREH5EXQ4DSZGAOJFX6PMJRYLQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.420790778 (602h22m53.304204413s) [noise=true , address=77.90.31.10:4519 , piece_id=FZLX3JO2323KSQ2LRF7ZQIJ54SKUQOFADMJ4RV4MLDCR7LAKYLGA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421151603 (602h22m53.304565232s) [noise=true , address=77.90.31.18:4805 , piece_id=MEPMORRPK3YN2CH6I5CCXPWVKS2J2FLFWBZYXZPGPKTWLMVWO7UA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421473878 (602h22m53.304887505s) [noise=true , address=45.91.116.5:4299 , piece_id=6MPSMO2NGZF7KUKW62F6DV6MB7EUCUSBS37JTQWW4CFCFL6R5WUA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421830327 (602h22m53.305243957s) [noise=true , address=45.91.116.9:4261 , piece_id=RJPIMYLOKVTWT7OAPDSLZRDKAS6S24MDJJEDO7R6T6KES6QOTKJQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422004819 (602h22m53.305418454s) [seg_pos={0 0}, upload=481619]: Done begin upload segment. Jul 18 11:33:53.422067705 (602h22m53.305481341s) [noise=true , address=45.91.116.13:4154 , piece_id=3DWHEX64WIUEFNTA7OQSI3N246VV5PL656S4BBZLKZRRFSYYPBSA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422255208 (602h22m53.305668836s) [noise=true , address=77.90.31.14:4662 , piece_id=3QLWYPH4XRRGSGS2TWRLD33ZYGAWGQTUOHVRNARWKSFLAXDD5V6A, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422510516 (602h22m53.305924152s) [noise=true , address=45.91.116.5:4298 , piece_id=MJCI34UOQBT7QVK5DJU35742ISYLWFUXYZKNZB2JDY2VVTVRNSKA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422891686 (602h22m53.306305323s) [noise=true , address=104.36.82.14:4223 , piece_id=HTLVOCDVQC25FPTK47EPLVESUQF4SQBARDGX34XF4H44S2KHR4KA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.423286438 (602h22m53.306700076s) [noise=true , address=77.90.31.16:4733 , piece_id=7GKK3MIAQZE55KUZ7CISUAKVOIY253FA5X3IPOEB46KIGZYHMQGQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.423650064 (602h22m53.307063701s) [noise=true , address=77.90.31.11:4555 , piece_id=36G5A5ZFW63MBSXUI5PUMXRWQWMIRUYB57WKV5MSFHIPOTWNGUQA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424001413 (602h22m53.30741505s) [noise=true , address=45.91.116.8:4047 , piece_id=TH7446RXMI3N2KVGWO4RFEJRJT7VNQK3BCOV4LX3NPOKFUPDLPAQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424340658 (602h22m53.307754287s) [noise=true , address=104.36.82.8:4971 , piece_id=GDLNL7U7CWXP32IIHOCDOCOFU5LFCQIOUE7Y5JR27BYGSGPBH5DQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424783800 (602h22m53.308197433s) [noise=true , address=104.36.82.4:4801 , piece_id=DUDK4ODWMXKWOX3T44XQANH7K6IDJY5XUYAZAD6URYVQVD2EGGNA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:54.037250564 (602h22m53.920664177s) [upload=481619]: Next returned nil segment Jul 18 11:33:54.037268249 (602h22m53.920681862s) [upload=481619]: Waiting for error group managing segments... Jul 18 11:33:54.174223271 (602h22m54.057636898s) [noise=true , address=45.91.116.5:4299 , piece_id=6MPSMO2NGZF7KUKW62F6DV6MB7EUCUSBS37JTQWW4CFCFL6R5WUA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.175017283 (602h22m54.058430971s) [noise=true , address=45.91.116.13:4154 , piece_id=3DWHEX64WIUEFNTA7OQSI3N246VV5PL656S4BBZLKZRRFSYYPBSA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.186693354 (602h22m54.070106972s) [noise=true , address=77.90.31.10:4519 , piece_id=FZLX3JO2323KSQ2LRF7ZQIJ54SKUQOFADMJ4RV4MLDCR7LAKYLGA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.194210810 (602h22m54.077624423s) [noise=true , address=45.91.116.12:4227 , piece_id=YZHWNLTXU4C7U6BD4ZMOASYHRKWOJX4SHTYSZCBFSCUCSIQGWKOA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.194454877 (602h22m54.077868504s) [noise=true , address=45.91.116.9:4261 , piece_id=RJPIMYLOKVTWT7OAPDSLZRDKAS6S24MDJJEDO7R6T6KES6QOTKJQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.198427713 (602h22m54.08184133s) [noise=true , address=45.91.116.5:4298 , piece_id=MJCI34UOQBT7QVK5DJU35742ISYLWFUXYZKNZB2JDY2VVTVRNSKA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.198472131 (602h22m54.081885751s) [noise=true , address=45.91.116.8:4047 , piece_id=TH7446RXMI3N2KVGWO4RFEJRJT7VNQK3BCOV4LX3NPOKFUPDLPAQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.207083885 (602h22m54.090497512s) [noise=true , address=104.36.82.4:4801 , piece_id=DUDK4ODWMXKWOX3T44XQANH7K6IDJY5XUYAZAD6URYVQVD2EGGNA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.208816335 (602h22m54.09222996s) [noise=true , address=104.36.82.8:4971 , piece_id=GDLNL7U7CWXP32IIHOCDOCOFU5LFCQIOUE7Y5JR27BYGSGPBH5DQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.336523319 (602h22m54.219936957s) [noise=true , address=104.36.82.14:4223 , piece_id=HTLVOCDVQC25FPTK47EPLVESUQF4SQBARDGX34XF4H44S2KHR4KA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:55.287929075 (602h22m55.171342693s) [noise=true , address=45.91.116.10:4082 , piece_id=MTW24CAAS7GMDLOJPXNHI6GEAZMREH5EXQ4DSZGAOJFX6PMJRYLQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:34:23.031611818 (602h23m22.915025445s) [noise=true , address=77.90.31.18:4805 , piece_id=MEPMORRPK3YN2CH6I5CCXPWVKS2J2FLFWBZYXZPGPKTWLMVWO7UA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: rpc: dial tcp 77.90.31.18:4805: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031747683 (602h23m22.915161316s) [noise=true , address=77.90.31.11:4555 , piece_id=36G5A5ZFW63MBSXUI5PUMXRWQWMIRUYB57WKV5MSFHIPOTWNGUQA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 139.59.144.22:43832->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031825318 (602h23m22.915238945s) [noise=true , address=77.90.31.16:4733 , piece_id=7GKK3MIAQZE55KUZ7CISUAKVOIY253FA5X3IPOEB46KIGZYHMQGQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:37494->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031906140 (602h23m22.915319768s) [noise=true , address=77.90.31.15:4699 , piece_id=YJBX34SLFMFNBBFJA3G2KDKB4HGIWY743XTQ5OHVFE5RJUDKYLXA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:45528->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031982671 (602h23m22.9153963s) [noise=true , address=77.90.31.14:4662 , piece_id=3QLWYPH4XRRGSGS2TWRLD33ZYGAWGQTUOHVRNARWKSFLAXDD5V6A, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:44356->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.032005226 (602h23m22.915418863s) [seg_pos={0 0}, upload=481619]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 11:34:23.158376157 (602h23m23.041789785s) [upload=481619]: Done uploading segments... Jul 18 11:34:23.158427828 (602h23m23.041841454s) [upload=481619]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.362697984s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.745303524s FAIL (2025-07-18 11:38:48 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.745343438s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 1.045707669s FAIL (2025-07-18 11:38:43 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.045728201s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 1.127877899s FAIL (2025-07-18 11:38:14 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.127898848s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.......X.....X.X..........X...............X..........X......] duration: 2.333332573s FAIL (2025-07-18 11:37:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== , upload=481608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:32:54.154838589 (602h21m54.038252206s) [upload=481608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:32:54.154843826 (602h21m54.038257453s) [upload=481608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:32:54.305725283 (602h21m54.189138898s) [upload=481608]: Done uploading segments... Jul 18 11:32:54.305745379 (602h21m54.189159007s) [upload=481608]: Upload finished. err: <nil> Jul 18 11:33:53.306124309 (602h22m53.189537928s) [upload=481619]: Starting upload Jul 18 11:33:53.306205552 (602h22m53.18961919s) [upload=481619]: Done starting upload Jul 18 11:33:53.306275544 (602h22m53.189689161s) [upload=481619]: Uploading segments... Jul 18 11:33:53.306475959 (602h22m53.189889586s) [upload=481619]: Got next segment. Inline: false Jul 18 11:33:53.306513703 (602h22m53.18992732s) [upload=481619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:33:53.306519968 (602h22m53.189933586s) [upload=481619]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:33:53.419183467 (602h22m53.302597096s) [seg_pos={0 0}, upload=481619]: Begin upload segment... Jul 18 11:33:53.419316113 (602h22m53.302729751s) [noise=true , address=45.91.116.12:4227 , piece_id=YZHWNLTXU4C7U6BD4ZMOASYHRKWOJX4SHTYSZCBFSCUCSIQGWKOA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.419850102 (602h22m53.303263734s) [noise=true , address=77.90.31.15:4699 , piece_id=YJBX34SLFMFNBBFJA3G2KDKB4HGIWY743XTQ5OHVFE5RJUDKYLXA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.420226364 (602h22m53.30364s) [noise=true , address=45.91.116.10:4082 , piece_id=MTW24CAAS7GMDLOJPXNHI6GEAZMREH5EXQ4DSZGAOJFX6PMJRYLQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.420790778 (602h22m53.304204413s) [noise=true , address=77.90.31.10:4519 , piece_id=FZLX3JO2323KSQ2LRF7ZQIJ54SKUQOFADMJ4RV4MLDCR7LAKYLGA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421151603 (602h22m53.304565232s) [noise=true , address=77.90.31.18:4805 , piece_id=MEPMORRPK3YN2CH6I5CCXPWVKS2J2FLFWBZYXZPGPKTWLMVWO7UA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421473878 (602h22m53.304887505s) [noise=true , address=45.91.116.5:4299 , piece_id=6MPSMO2NGZF7KUKW62F6DV6MB7EUCUSBS37JTQWW4CFCFL6R5WUA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421830327 (602h22m53.305243957s) [noise=true , address=45.91.116.9:4261 , piece_id=RJPIMYLOKVTWT7OAPDSLZRDKAS6S24MDJJEDO7R6T6KES6QOTKJQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422004819 (602h22m53.305418454s) [seg_pos={0 0}, upload=481619]: Done begin upload segment. Jul 18 11:33:53.422067705 (602h22m53.305481341s) [noise=true , address=45.91.116.13:4154 , piece_id=3DWHEX64WIUEFNTA7OQSI3N246VV5PL656S4BBZLKZRRFSYYPBSA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422255208 (602h22m53.305668836s) [noise=true , address=77.90.31.14:4662 , piece_id=3QLWYPH4XRRGSGS2TWRLD33ZYGAWGQTUOHVRNARWKSFLAXDD5V6A, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422510516 (602h22m53.305924152s) [noise=true , address=45.91.116.5:4298 , piece_id=MJCI34UOQBT7QVK5DJU35742ISYLWFUXYZKNZB2JDY2VVTVRNSKA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422891686 (602h22m53.306305323s) [noise=true , address=104.36.82.14:4223 , piece_id=HTLVOCDVQC25FPTK47EPLVESUQF4SQBARDGX34XF4H44S2KHR4KA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.423286438 (602h22m53.306700076s) [noise=true , address=77.90.31.16:4733 , piece_id=7GKK3MIAQZE55KUZ7CISUAKVOIY253FA5X3IPOEB46KIGZYHMQGQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.423650064 (602h22m53.307063701s) [noise=true , address=77.90.31.11:4555 , piece_id=36G5A5ZFW63MBSXUI5PUMXRWQWMIRUYB57WKV5MSFHIPOTWNGUQA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424001413 (602h22m53.30741505s) [noise=true , address=45.91.116.8:4047 , piece_id=TH7446RXMI3N2KVGWO4RFEJRJT7VNQK3BCOV4LX3NPOKFUPDLPAQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424340658 (602h22m53.307754287s) [noise=true , address=104.36.82.8:4971 , piece_id=GDLNL7U7CWXP32IIHOCDOCOFU5LFCQIOUE7Y5JR27BYGSGPBH5DQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424783800 (602h22m53.308197433s) [noise=true , address=104.36.82.4:4801 , piece_id=DUDK4ODWMXKWOX3T44XQANH7K6IDJY5XUYAZAD6URYVQVD2EGGNA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:54.037250564 (602h22m53.920664177s) [upload=481619]: Next returned nil segment Jul 18 11:33:54.037268249 (602h22m53.920681862s) [upload=481619]: Waiting for error group managing segments... Jul 18 11:33:54.174223271 (602h22m54.057636898s) [noise=true , address=45.91.116.5:4299 , piece_id=6MPSMO2NGZF7KUKW62F6DV6MB7EUCUSBS37JTQWW4CFCFL6R5WUA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.175017283 (602h22m54.058430971s) [noise=true , address=45.91.116.13:4154 , piece_id=3DWHEX64WIUEFNTA7OQSI3N246VV5PL656S4BBZLKZRRFSYYPBSA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.186693354 (602h22m54.070106972s) [noise=true , address=77.90.31.10:4519 , piece_id=FZLX3JO2323KSQ2LRF7ZQIJ54SKUQOFADMJ4RV4MLDCR7LAKYLGA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.194210810 (602h22m54.077624423s) [noise=true , address=45.91.116.12:4227 , piece_id=YZHWNLTXU4C7U6BD4ZMOASYHRKWOJX4SHTYSZCBFSCUCSIQGWKOA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.194454877 (602h22m54.077868504s) [noise=true , address=45.91.116.9:4261 , piece_id=RJPIMYLOKVTWT7OAPDSLZRDKAS6S24MDJJEDO7R6T6KES6QOTKJQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.198427713 (602h22m54.08184133s) [noise=true , address=45.91.116.5:4298 , piece_id=MJCI34UOQBT7QVK5DJU35742ISYLWFUXYZKNZB2JDY2VVTVRNSKA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.198472131 (602h22m54.081885751s) [noise=true , address=45.91.116.8:4047 , piece_id=TH7446RXMI3N2KVGWO4RFEJRJT7VNQK3BCOV4LX3NPOKFUPDLPAQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.207083885 (602h22m54.090497512s) [noise=true , address=104.36.82.4:4801 , piece_id=DUDK4ODWMXKWOX3T44XQANH7K6IDJY5XUYAZAD6URYVQVD2EGGNA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.208816335 (602h22m54.09222996s) [noise=true , address=104.36.82.8:4971 , piece_id=GDLNL7U7CWXP32IIHOCDOCOFU5LFCQIOUE7Y5JR27BYGSGPBH5DQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.336523319 (602h22m54.219936957s) [noise=true , address=104.36.82.14:4223 , piece_id=HTLVOCDVQC25FPTK47EPLVESUQF4SQBARDGX34XF4H44S2KHR4KA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:55.287929075 (602h22m55.171342693s) [noise=true , address=45.91.116.10:4082 , piece_id=MTW24CAAS7GMDLOJPXNHI6GEAZMREH5EXQ4DSZGAOJFX6PMJRYLQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:34:23.031611818 (602h23m22.915025445s) [noise=true , address=77.90.31.18:4805 , piece_id=MEPMORRPK3YN2CH6I5CCXPWVKS2J2FLFWBZYXZPGPKTWLMVWO7UA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: rpc: dial tcp 77.90.31.18:4805: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031747683 (602h23m22.915161316s) [noise=true , address=77.90.31.11:4555 , piece_id=36G5A5ZFW63MBSXUI5PUMXRWQWMIRUYB57WKV5MSFHIPOTWNGUQA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 139.59.144.22:43832->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031825318 (602h23m22.915238945s) [noise=true , address=77.90.31.16:4733 , piece_id=7GKK3MIAQZE55KUZ7CISUAKVOIY253FA5X3IPOEB46KIGZYHMQGQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:37494->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031906140 (602h23m22.915319768s) [noise=true , address=77.90.31.15:4699 , piece_id=YJBX34SLFMFNBBFJA3G2KDKB4HGIWY743XTQ5OHVFE5RJUDKYLXA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:45528->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031982671 (602h23m22.9153963s) [noise=true , address=77.90.31.14:4662 , piece_id=3QLWYPH4XRRGSGS2TWRLD33ZYGAWGQTUOHVRNARWKSFLAXDD5V6A, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:44356->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.032005226 (602h23m22.915418863s) [seg_pos={0 0}, upload=481619]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 11:34:23.158376157 (602h23m23.041789785s) [upload=481619]: Done uploading segments... Jul 18 11:34:23.158427828 (602h23m23.041841454s) [upload=481619]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.333385073s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 1.103208766s FAIL (2025-07-18 11:37:51 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.1032417s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.533217007s FAIL (2025-07-18 11:37:47 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.533257766s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [........X.....X.X..........X...............X..........X.....] duration: 2.448877875s FAIL (2025-07-18 11:37:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== , upload=481608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:32:54.154838589 (602h21m54.038252206s) [upload=481608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:32:54.154843826 (602h21m54.038257453s) [upload=481608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:32:54.305725283 (602h21m54.189138898s) [upload=481608]: Done uploading segments... Jul 18 11:32:54.305745379 (602h21m54.189159007s) [upload=481608]: Upload finished. err: <nil> Jul 18 11:33:53.306124309 (602h22m53.189537928s) [upload=481619]: Starting upload Jul 18 11:33:53.306205552 (602h22m53.18961919s) [upload=481619]: Done starting upload Jul 18 11:33:53.306275544 (602h22m53.189689161s) [upload=481619]: Uploading segments... Jul 18 11:33:53.306475959 (602h22m53.189889586s) [upload=481619]: Got next segment. Inline: false Jul 18 11:33:53.306513703 (602h22m53.18992732s) [upload=481619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:33:53.306519968 (602h22m53.189933586s) [upload=481619]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:33:53.419183467 (602h22m53.302597096s) [seg_pos={0 0}, upload=481619]: Begin upload segment... Jul 18 11:33:53.419316113 (602h22m53.302729751s) [noise=true , address=45.91.116.12:4227 , piece_id=YZHWNLTXU4C7U6BD4ZMOASYHRKWOJX4SHTYSZCBFSCUCSIQGWKOA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.419850102 (602h22m53.303263734s) [noise=true , address=77.90.31.15:4699 , piece_id=YJBX34SLFMFNBBFJA3G2KDKB4HGIWY743XTQ5OHVFE5RJUDKYLXA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.420226364 (602h22m53.30364s) [noise=true , address=45.91.116.10:4082 , piece_id=MTW24CAAS7GMDLOJPXNHI6GEAZMREH5EXQ4DSZGAOJFX6PMJRYLQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.420790778 (602h22m53.304204413s) [noise=true , address=77.90.31.10:4519 , piece_id=FZLX3JO2323KSQ2LRF7ZQIJ54SKUQOFADMJ4RV4MLDCR7LAKYLGA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421151603 (602h22m53.304565232s) [noise=true , address=77.90.31.18:4805 , piece_id=MEPMORRPK3YN2CH6I5CCXPWVKS2J2FLFWBZYXZPGPKTWLMVWO7UA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421473878 (602h22m53.304887505s) [noise=true , address=45.91.116.5:4299 , piece_id=6MPSMO2NGZF7KUKW62F6DV6MB7EUCUSBS37JTQWW4CFCFL6R5WUA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421830327 (602h22m53.305243957s) [noise=true , address=45.91.116.9:4261 , piece_id=RJPIMYLOKVTWT7OAPDSLZRDKAS6S24MDJJEDO7R6T6KES6QOTKJQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422004819 (602h22m53.305418454s) [seg_pos={0 0}, upload=481619]: Done begin upload segment. Jul 18 11:33:53.422067705 (602h22m53.305481341s) [noise=true , address=45.91.116.13:4154 , piece_id=3DWHEX64WIUEFNTA7OQSI3N246VV5PL656S4BBZLKZRRFSYYPBSA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422255208 (602h22m53.305668836s) [noise=true , address=77.90.31.14:4662 , piece_id=3QLWYPH4XRRGSGS2TWRLD33ZYGAWGQTUOHVRNARWKSFLAXDD5V6A, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422510516 (602h22m53.305924152s) [noise=true , address=45.91.116.5:4298 , piece_id=MJCI34UOQBT7QVK5DJU35742ISYLWFUXYZKNZB2JDY2VVTVRNSKA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422891686 (602h22m53.306305323s) [noise=true , address=104.36.82.14:4223 , piece_id=HTLVOCDVQC25FPTK47EPLVESUQF4SQBARDGX34XF4H44S2KHR4KA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.423286438 (602h22m53.306700076s) [noise=true , address=77.90.31.16:4733 , piece_id=7GKK3MIAQZE55KUZ7CISUAKVOIY253FA5X3IPOEB46KIGZYHMQGQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.423650064 (602h22m53.307063701s) [noise=true , address=77.90.31.11:4555 , piece_id=36G5A5ZFW63MBSXUI5PUMXRWQWMIRUYB57WKV5MSFHIPOTWNGUQA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424001413 (602h22m53.30741505s) [noise=true , address=45.91.116.8:4047 , piece_id=TH7446RXMI3N2KVGWO4RFEJRJT7VNQK3BCOV4LX3NPOKFUPDLPAQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424340658 (602h22m53.307754287s) [noise=true , address=104.36.82.8:4971 , piece_id=GDLNL7U7CWXP32IIHOCDOCOFU5LFCQIOUE7Y5JR27BYGSGPBH5DQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424783800 (602h22m53.308197433s) [noise=true , address=104.36.82.4:4801 , piece_id=DUDK4ODWMXKWOX3T44XQANH7K6IDJY5XUYAZAD6URYVQVD2EGGNA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:54.037250564 (602h22m53.920664177s) [upload=481619]: Next returned nil segment Jul 18 11:33:54.037268249 (602h22m53.920681862s) [upload=481619]: Waiting for error group managing segments... Jul 18 11:33:54.174223271 (602h22m54.057636898s) [noise=true , address=45.91.116.5:4299 , piece_id=6MPSMO2NGZF7KUKW62F6DV6MB7EUCUSBS37JTQWW4CFCFL6R5WUA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.175017283 (602h22m54.058430971s) [noise=true , address=45.91.116.13:4154 , piece_id=3DWHEX64WIUEFNTA7OQSI3N246VV5PL656S4BBZLKZRRFSYYPBSA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.186693354 (602h22m54.070106972s) [noise=true , address=77.90.31.10:4519 , piece_id=FZLX3JO2323KSQ2LRF7ZQIJ54SKUQOFADMJ4RV4MLDCR7LAKYLGA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.194210810 (602h22m54.077624423s) [noise=true , address=45.91.116.12:4227 , piece_id=YZHWNLTXU4C7U6BD4ZMOASYHRKWOJX4SHTYSZCBFSCUCSIQGWKOA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.194454877 (602h22m54.077868504s) [noise=true , address=45.91.116.9:4261 , piece_id=RJPIMYLOKVTWT7OAPDSLZRDKAS6S24MDJJEDO7R6T6KES6QOTKJQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.198427713 (602h22m54.08184133s) [noise=true , address=45.91.116.5:4298 , piece_id=MJCI34UOQBT7QVK5DJU35742ISYLWFUXYZKNZB2JDY2VVTVRNSKA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.198472131 (602h22m54.081885751s) [noise=true , address=45.91.116.8:4047 , piece_id=TH7446RXMI3N2KVGWO4RFEJRJT7VNQK3BCOV4LX3NPOKFUPDLPAQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.207083885 (602h22m54.090497512s) [noise=true , address=104.36.82.4:4801 , piece_id=DUDK4ODWMXKWOX3T44XQANH7K6IDJY5XUYAZAD6URYVQVD2EGGNA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.208816335 (602h22m54.09222996s) [noise=true , address=104.36.82.8:4971 , piece_id=GDLNL7U7CWXP32IIHOCDOCOFU5LFCQIOUE7Y5JR27BYGSGPBH5DQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.336523319 (602h22m54.219936957s) [noise=true , address=104.36.82.14:4223 , piece_id=HTLVOCDVQC25FPTK47EPLVESUQF4SQBARDGX34XF4H44S2KHR4KA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:55.287929075 (602h22m55.171342693s) [noise=true , address=45.91.116.10:4082 , piece_id=MTW24CAAS7GMDLOJPXNHI6GEAZMREH5EXQ4DSZGAOJFX6PMJRYLQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:34:23.031611818 (602h23m22.915025445s) [noise=true , address=77.90.31.18:4805 , piece_id=MEPMORRPK3YN2CH6I5CCXPWVKS2J2FLFWBZYXZPGPKTWLMVWO7UA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: rpc: dial tcp 77.90.31.18:4805: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031747683 (602h23m22.915161316s) [noise=true , address=77.90.31.11:4555 , piece_id=36G5A5ZFW63MBSXUI5PUMXRWQWMIRUYB57WKV5MSFHIPOTWNGUQA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 139.59.144.22:43832->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031825318 (602h23m22.915238945s) [noise=true , address=77.90.31.16:4733 , piece_id=7GKK3MIAQZE55KUZ7CISUAKVOIY253FA5X3IPOEB46KIGZYHMQGQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:37494->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031906140 (602h23m22.915319768s) [noise=true , address=77.90.31.15:4699 , piece_id=YJBX34SLFMFNBBFJA3G2KDKB4HGIWY743XTQ5OHVFE5RJUDKYLXA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:45528->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031982671 (602h23m22.9153963s) [noise=true , address=77.90.31.14:4662 , piece_id=3QLWYPH4XRRGSGS2TWRLD33ZYGAWGQTUOHVRNARWKSFLAXDD5V6A, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:44356->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.032005226 (602h23m22.915418863s) [seg_pos={0 0}, upload=481619]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 11:34:23.158376157 (602h23m23.041789785s) [upload=481619]: Done uploading segments... Jul 18 11:34:23.158427828 (602h23m23.041841454s) [upload=481619]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.448969631s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 3.183289547s FAIL (2025-07-18 11:36:42 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 3.18334867s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 1.058567846s FAIL (2025-07-18 11:36:41 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.058598624s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.........X.....X.X..........X...............X..........X....] duration: 2.399783417s FAIL (2025-07-18 11:36:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== , upload=481608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:32:54.154838589 (602h21m54.038252206s) [upload=481608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:32:54.154843826 (602h21m54.038257453s) [upload=481608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:32:54.305725283 (602h21m54.189138898s) [upload=481608]: Done uploading segments... Jul 18 11:32:54.305745379 (602h21m54.189159007s) [upload=481608]: Upload finished. err: <nil> Jul 18 11:33:53.306124309 (602h22m53.189537928s) [upload=481619]: Starting upload Jul 18 11:33:53.306205552 (602h22m53.18961919s) [upload=481619]: Done starting upload Jul 18 11:33:53.306275544 (602h22m53.189689161s) [upload=481619]: Uploading segments... Jul 18 11:33:53.306475959 (602h22m53.189889586s) [upload=481619]: Got next segment. Inline: false Jul 18 11:33:53.306513703 (602h22m53.18992732s) [upload=481619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:33:53.306519968 (602h22m53.189933586s) [upload=481619]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:33:53.419183467 (602h22m53.302597096s) [seg_pos={0 0}, upload=481619]: Begin upload segment... Jul 18 11:33:53.419316113 (602h22m53.302729751s) [noise=true , address=45.91.116.12:4227 , piece_id=YZHWNLTXU4C7U6BD4ZMOASYHRKWOJX4SHTYSZCBFSCUCSIQGWKOA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.419850102 (602h22m53.303263734s) [noise=true , address=77.90.31.15:4699 , piece_id=YJBX34SLFMFNBBFJA3G2KDKB4HGIWY743XTQ5OHVFE5RJUDKYLXA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.420226364 (602h22m53.30364s) [noise=true , address=45.91.116.10:4082 , piece_id=MTW24CAAS7GMDLOJPXNHI6GEAZMREH5EXQ4DSZGAOJFX6PMJRYLQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.420790778 (602h22m53.304204413s) [noise=true , address=77.90.31.10:4519 , piece_id=FZLX3JO2323KSQ2LRF7ZQIJ54SKUQOFADMJ4RV4MLDCR7LAKYLGA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421151603 (602h22m53.304565232s) [noise=true , address=77.90.31.18:4805 , piece_id=MEPMORRPK3YN2CH6I5CCXPWVKS2J2FLFWBZYXZPGPKTWLMVWO7UA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421473878 (602h22m53.304887505s) [noise=true , address=45.91.116.5:4299 , piece_id=6MPSMO2NGZF7KUKW62F6DV6MB7EUCUSBS37JTQWW4CFCFL6R5WUA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421830327 (602h22m53.305243957s) [noise=true , address=45.91.116.9:4261 , piece_id=RJPIMYLOKVTWT7OAPDSLZRDKAS6S24MDJJEDO7R6T6KES6QOTKJQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422004819 (602h22m53.305418454s) [seg_pos={0 0}, upload=481619]: Done begin upload segment. Jul 18 11:33:53.422067705 (602h22m53.305481341s) [noise=true , address=45.91.116.13:4154 , piece_id=3DWHEX64WIUEFNTA7OQSI3N246VV5PL656S4BBZLKZRRFSYYPBSA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422255208 (602h22m53.305668836s) [noise=true , address=77.90.31.14:4662 , piece_id=3QLWYPH4XRRGSGS2TWRLD33ZYGAWGQTUOHVRNARWKSFLAXDD5V6A, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422510516 (602h22m53.305924152s) [noise=true , address=45.91.116.5:4298 , piece_id=MJCI34UOQBT7QVK5DJU35742ISYLWFUXYZKNZB2JDY2VVTVRNSKA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422891686 (602h22m53.306305323s) [noise=true , address=104.36.82.14:4223 , piece_id=HTLVOCDVQC25FPTK47EPLVESUQF4SQBARDGX34XF4H44S2KHR4KA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.423286438 (602h22m53.306700076s) [noise=true , address=77.90.31.16:4733 , piece_id=7GKK3MIAQZE55KUZ7CISUAKVOIY253FA5X3IPOEB46KIGZYHMQGQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.423650064 (602h22m53.307063701s) [noise=true , address=77.90.31.11:4555 , piece_id=36G5A5ZFW63MBSXUI5PUMXRWQWMIRUYB57WKV5MSFHIPOTWNGUQA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424001413 (602h22m53.30741505s) [noise=true , address=45.91.116.8:4047 , piece_id=TH7446RXMI3N2KVGWO4RFEJRJT7VNQK3BCOV4LX3NPOKFUPDLPAQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424340658 (602h22m53.307754287s) [noise=true , address=104.36.82.8:4971 , piece_id=GDLNL7U7CWXP32IIHOCDOCOFU5LFCQIOUE7Y5JR27BYGSGPBH5DQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424783800 (602h22m53.308197433s) [noise=true , address=104.36.82.4:4801 , piece_id=DUDK4ODWMXKWOX3T44XQANH7K6IDJY5XUYAZAD6URYVQVD2EGGNA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:54.037250564 (602h22m53.920664177s) [upload=481619]: Next returned nil segment Jul 18 11:33:54.037268249 (602h22m53.920681862s) [upload=481619]: Waiting for error group managing segments... Jul 18 11:33:54.174223271 (602h22m54.057636898s) [noise=true , address=45.91.116.5:4299 , piece_id=6MPSMO2NGZF7KUKW62F6DV6MB7EUCUSBS37JTQWW4CFCFL6R5WUA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.175017283 (602h22m54.058430971s) [noise=true , address=45.91.116.13:4154 , piece_id=3DWHEX64WIUEFNTA7OQSI3N246VV5PL656S4BBZLKZRRFSYYPBSA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.186693354 (602h22m54.070106972s) [noise=true , address=77.90.31.10:4519 , piece_id=FZLX3JO2323KSQ2LRF7ZQIJ54SKUQOFADMJ4RV4MLDCR7LAKYLGA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.194210810 (602h22m54.077624423s) [noise=true , address=45.91.116.12:4227 , piece_id=YZHWNLTXU4C7U6BD4ZMOASYHRKWOJX4SHTYSZCBFSCUCSIQGWKOA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.194454877 (602h22m54.077868504s) [noise=true , address=45.91.116.9:4261 , piece_id=RJPIMYLOKVTWT7OAPDSLZRDKAS6S24MDJJEDO7R6T6KES6QOTKJQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.198427713 (602h22m54.08184133s) [noise=true , address=45.91.116.5:4298 , piece_id=MJCI34UOQBT7QVK5DJU35742ISYLWFUXYZKNZB2JDY2VVTVRNSKA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.198472131 (602h22m54.081885751s) [noise=true , address=45.91.116.8:4047 , piece_id=TH7446RXMI3N2KVGWO4RFEJRJT7VNQK3BCOV4LX3NPOKFUPDLPAQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.207083885 (602h22m54.090497512s) [noise=true , address=104.36.82.4:4801 , piece_id=DUDK4ODWMXKWOX3T44XQANH7K6IDJY5XUYAZAD6URYVQVD2EGGNA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.208816335 (602h22m54.09222996s) [noise=true , address=104.36.82.8:4971 , piece_id=GDLNL7U7CWXP32IIHOCDOCOFU5LFCQIOUE7Y5JR27BYGSGPBH5DQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.336523319 (602h22m54.219936957s) [noise=true , address=104.36.82.14:4223 , piece_id=HTLVOCDVQC25FPTK47EPLVESUQF4SQBARDGX34XF4H44S2KHR4KA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:55.287929075 (602h22m55.171342693s) [noise=true , address=45.91.116.10:4082 , piece_id=MTW24CAAS7GMDLOJPXNHI6GEAZMREH5EXQ4DSZGAOJFX6PMJRYLQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:34:23.031611818 (602h23m22.915025445s) [noise=true , address=77.90.31.18:4805 , piece_id=MEPMORRPK3YN2CH6I5CCXPWVKS2J2FLFWBZYXZPGPKTWLMVWO7UA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: rpc: dial tcp 77.90.31.18:4805: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031747683 (602h23m22.915161316s) [noise=true , address=77.90.31.11:4555 , piece_id=36G5A5ZFW63MBSXUI5PUMXRWQWMIRUYB57WKV5MSFHIPOTWNGUQA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 139.59.144.22:43832->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031825318 (602h23m22.915238945s) [noise=true , address=77.90.31.16:4733 , piece_id=7GKK3MIAQZE55KUZ7CISUAKVOIY253FA5X3IPOEB46KIGZYHMQGQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:37494->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031906140 (602h23m22.915319768s) [noise=true , address=77.90.31.15:4699 , piece_id=YJBX34SLFMFNBBFJA3G2KDKB4HGIWY743XTQ5OHVFE5RJUDKYLXA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:45528->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031982671 (602h23m22.9153963s) [noise=true , address=77.90.31.14:4662 , piece_id=3QLWYPH4XRRGSGS2TWRLD33ZYGAWGQTUOHVRNARWKSFLAXDD5V6A, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:44356->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.032005226 (602h23m22.915418863s) [seg_pos={0 0}, upload=481619]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 11:34:23.158376157 (602h23m23.041789785s) [upload=481619]: Done uploading segments... Jul 18 11:34:23.158427828 (602h23m23.041841454s) [upload=481619]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.399818377s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.917857614s FAIL (2025-07-18 11:36:21 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.917889639s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 1.408231421s FAIL (2025-07-18 11:36:03 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.408388056s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.647197599s FAIL (2025-07-18 11:36:01 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.647233129s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........X.....X.X..........X...............X..........X...] duration: 2.377352575s FAIL (2025-07-18 11:35:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== , upload=481608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:32:54.154838589 (602h21m54.038252206s) [upload=481608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:32:54.154843826 (602h21m54.038257453s) [upload=481608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:32:54.305725283 (602h21m54.189138898s) [upload=481608]: Done uploading segments... Jul 18 11:32:54.305745379 (602h21m54.189159007s) [upload=481608]: Upload finished. err: <nil> Jul 18 11:33:53.306124309 (602h22m53.189537928s) [upload=481619]: Starting upload Jul 18 11:33:53.306205552 (602h22m53.18961919s) [upload=481619]: Done starting upload Jul 18 11:33:53.306275544 (602h22m53.189689161s) [upload=481619]: Uploading segments... Jul 18 11:33:53.306475959 (602h22m53.189889586s) [upload=481619]: Got next segment. Inline: false Jul 18 11:33:53.306513703 (602h22m53.18992732s) [upload=481619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:33:53.306519968 (602h22m53.189933586s) [upload=481619]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:33:53.419183467 (602h22m53.302597096s) [seg_pos={0 0}, upload=481619]: Begin upload segment... Jul 18 11:33:53.419316113 (602h22m53.302729751s) [noise=true , address=45.91.116.12:4227 , piece_id=YZHWNLTXU4C7U6BD4ZMOASYHRKWOJX4SHTYSZCBFSCUCSIQGWKOA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.419850102 (602h22m53.303263734s) [noise=true , address=77.90.31.15:4699 , piece_id=YJBX34SLFMFNBBFJA3G2KDKB4HGIWY743XTQ5OHVFE5RJUDKYLXA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.420226364 (602h22m53.30364s) [noise=true , address=45.91.116.10:4082 , piece_id=MTW24CAAS7GMDLOJPXNHI6GEAZMREH5EXQ4DSZGAOJFX6PMJRYLQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.420790778 (602h22m53.304204413s) [noise=true , address=77.90.31.10:4519 , piece_id=FZLX3JO2323KSQ2LRF7ZQIJ54SKUQOFADMJ4RV4MLDCR7LAKYLGA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421151603 (602h22m53.304565232s) [noise=true , address=77.90.31.18:4805 , piece_id=MEPMORRPK3YN2CH6I5CCXPWVKS2J2FLFWBZYXZPGPKTWLMVWO7UA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421473878 (602h22m53.304887505s) [noise=true , address=45.91.116.5:4299 , piece_id=6MPSMO2NGZF7KUKW62F6DV6MB7EUCUSBS37JTQWW4CFCFL6R5WUA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421830327 (602h22m53.305243957s) [noise=true , address=45.91.116.9:4261 , piece_id=RJPIMYLOKVTWT7OAPDSLZRDKAS6S24MDJJEDO7R6T6KES6QOTKJQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422004819 (602h22m53.305418454s) [seg_pos={0 0}, upload=481619]: Done begin upload segment. Jul 18 11:33:53.422067705 (602h22m53.305481341s) [noise=true , address=45.91.116.13:4154 , piece_id=3DWHEX64WIUEFNTA7OQSI3N246VV5PL656S4BBZLKZRRFSYYPBSA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422255208 (602h22m53.305668836s) [noise=true , address=77.90.31.14:4662 , piece_id=3QLWYPH4XRRGSGS2TWRLD33ZYGAWGQTUOHVRNARWKSFLAXDD5V6A, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422510516 (602h22m53.305924152s) [noise=true , address=45.91.116.5:4298 , piece_id=MJCI34UOQBT7QVK5DJU35742ISYLWFUXYZKNZB2JDY2VVTVRNSKA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422891686 (602h22m53.306305323s) [noise=true , address=104.36.82.14:4223 , piece_id=HTLVOCDVQC25FPTK47EPLVESUQF4SQBARDGX34XF4H44S2KHR4KA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.423286438 (602h22m53.306700076s) [noise=true , address=77.90.31.16:4733 , piece_id=7GKK3MIAQZE55KUZ7CISUAKVOIY253FA5X3IPOEB46KIGZYHMQGQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.423650064 (602h22m53.307063701s) [noise=true , address=77.90.31.11:4555 , piece_id=36G5A5ZFW63MBSXUI5PUMXRWQWMIRUYB57WKV5MSFHIPOTWNGUQA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424001413 (602h22m53.30741505s) [noise=true , address=45.91.116.8:4047 , piece_id=TH7446RXMI3N2KVGWO4RFEJRJT7VNQK3BCOV4LX3NPOKFUPDLPAQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424340658 (602h22m53.307754287s) [noise=true , address=104.36.82.8:4971 , piece_id=GDLNL7U7CWXP32IIHOCDOCOFU5LFCQIOUE7Y5JR27BYGSGPBH5DQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424783800 (602h22m53.308197433s) [noise=true , address=104.36.82.4:4801 , piece_id=DUDK4ODWMXKWOX3T44XQANH7K6IDJY5XUYAZAD6URYVQVD2EGGNA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:54.037250564 (602h22m53.920664177s) [upload=481619]: Next returned nil segment Jul 18 11:33:54.037268249 (602h22m53.920681862s) [upload=481619]: Waiting for error group managing segments... Jul 18 11:33:54.174223271 (602h22m54.057636898s) [noise=true , address=45.91.116.5:4299 , piece_id=6MPSMO2NGZF7KUKW62F6DV6MB7EUCUSBS37JTQWW4CFCFL6R5WUA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.175017283 (602h22m54.058430971s) [noise=true , address=45.91.116.13:4154 , piece_id=3DWHEX64WIUEFNTA7OQSI3N246VV5PL656S4BBZLKZRRFSYYPBSA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.186693354 (602h22m54.070106972s) [noise=true , address=77.90.31.10:4519 , piece_id=FZLX3JO2323KSQ2LRF7ZQIJ54SKUQOFADMJ4RV4MLDCR7LAKYLGA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.194210810 (602h22m54.077624423s) [noise=true , address=45.91.116.12:4227 , piece_id=YZHWNLTXU4C7U6BD4ZMOASYHRKWOJX4SHTYSZCBFSCUCSIQGWKOA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.194454877 (602h22m54.077868504s) [noise=true , address=45.91.116.9:4261 , piece_id=RJPIMYLOKVTWT7OAPDSLZRDKAS6S24MDJJEDO7R6T6KES6QOTKJQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.198427713 (602h22m54.08184133s) [noise=true , address=45.91.116.5:4298 , piece_id=MJCI34UOQBT7QVK5DJU35742ISYLWFUXYZKNZB2JDY2VVTVRNSKA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.198472131 (602h22m54.081885751s) [noise=true , address=45.91.116.8:4047 , piece_id=TH7446RXMI3N2KVGWO4RFEJRJT7VNQK3BCOV4LX3NPOKFUPDLPAQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.207083885 (602h22m54.090497512s) [noise=true , address=104.36.82.4:4801 , piece_id=DUDK4ODWMXKWOX3T44XQANH7K6IDJY5XUYAZAD6URYVQVD2EGGNA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.208816335 (602h22m54.09222996s) [noise=true , address=104.36.82.8:4971 , piece_id=GDLNL7U7CWXP32IIHOCDOCOFU5LFCQIOUE7Y5JR27BYGSGPBH5DQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.336523319 (602h22m54.219936957s) [noise=true , address=104.36.82.14:4223 , piece_id=HTLVOCDVQC25FPTK47EPLVESUQF4SQBARDGX34XF4H44S2KHR4KA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:55.287929075 (602h22m55.171342693s) [noise=true , address=45.91.116.10:4082 , piece_id=MTW24CAAS7GMDLOJPXNHI6GEAZMREH5EXQ4DSZGAOJFX6PMJRYLQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:34:23.031611818 (602h23m22.915025445s) [noise=true , address=77.90.31.18:4805 , piece_id=MEPMORRPK3YN2CH6I5CCXPWVKS2J2FLFWBZYXZPGPKTWLMVWO7UA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: rpc: dial tcp 77.90.31.18:4805: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031747683 (602h23m22.915161316s) [noise=true , address=77.90.31.11:4555 , piece_id=36G5A5ZFW63MBSXUI5PUMXRWQWMIRUYB57WKV5MSFHIPOTWNGUQA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 139.59.144.22:43832->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031825318 (602h23m22.915238945s) [noise=true , address=77.90.31.16:4733 , piece_id=7GKK3MIAQZE55KUZ7CISUAKVOIY253FA5X3IPOEB46KIGZYHMQGQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:37494->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031906140 (602h23m22.915319768s) [noise=true , address=77.90.31.15:4699 , piece_id=YJBX34SLFMFNBBFJA3G2KDKB4HGIWY743XTQ5OHVFE5RJUDKYLXA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:45528->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031982671 (602h23m22.9153963s) [noise=true , address=77.90.31.14:4662 , piece_id=3QLWYPH4XRRGSGS2TWRLD33ZYGAWGQTUOHVRNARWKSFLAXDD5V6A, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:44356->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.032005226 (602h23m22.915418863s) [seg_pos={0 0}, upload=481619]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 11:34:23.158376157 (602h23m23.041789785s) [upload=481619]: Done uploading segments... Jul 18 11:34:23.158427828 (602h23m23.041841454s) [upload=481619]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.377382823s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.56640149s FAIL (2025-07-18 11:35:40 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.566435195s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........X.....X.X..........X...............X..........X..] duration: 2.432765108s FAIL (2025-07-18 11:35:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== , upload=481608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:32:54.154838589 (602h21m54.038252206s) [upload=481608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:32:54.154843826 (602h21m54.038257453s) [upload=481608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:32:54.305725283 (602h21m54.189138898s) [upload=481608]: Done uploading segments... Jul 18 11:32:54.305745379 (602h21m54.189159007s) [upload=481608]: Upload finished. err: <nil> Jul 18 11:33:53.306124309 (602h22m53.189537928s) [upload=481619]: Starting upload Jul 18 11:33:53.306205552 (602h22m53.18961919s) [upload=481619]: Done starting upload Jul 18 11:33:53.306275544 (602h22m53.189689161s) [upload=481619]: Uploading segments... Jul 18 11:33:53.306475959 (602h22m53.189889586s) [upload=481619]: Got next segment. Inline: false Jul 18 11:33:53.306513703 (602h22m53.18992732s) [upload=481619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:33:53.306519968 (602h22m53.189933586s) [upload=481619]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:33:53.419183467 (602h22m53.302597096s) [seg_pos={0 0}, upload=481619]: Begin upload segment... Jul 18 11:33:53.419316113 (602h22m53.302729751s) [noise=true , address=45.91.116.12:4227 , piece_id=YZHWNLTXU4C7U6BD4ZMOASYHRKWOJX4SHTYSZCBFSCUCSIQGWKOA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.419850102 (602h22m53.303263734s) [noise=true , address=77.90.31.15:4699 , piece_id=YJBX34SLFMFNBBFJA3G2KDKB4HGIWY743XTQ5OHVFE5RJUDKYLXA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.420226364 (602h22m53.30364s) [noise=true , address=45.91.116.10:4082 , piece_id=MTW24CAAS7GMDLOJPXNHI6GEAZMREH5EXQ4DSZGAOJFX6PMJRYLQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.420790778 (602h22m53.304204413s) [noise=true , address=77.90.31.10:4519 , piece_id=FZLX3JO2323KSQ2LRF7ZQIJ54SKUQOFADMJ4RV4MLDCR7LAKYLGA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421151603 (602h22m53.304565232s) [noise=true , address=77.90.31.18:4805 , piece_id=MEPMORRPK3YN2CH6I5CCXPWVKS2J2FLFWBZYXZPGPKTWLMVWO7UA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421473878 (602h22m53.304887505s) [noise=true , address=45.91.116.5:4299 , piece_id=6MPSMO2NGZF7KUKW62F6DV6MB7EUCUSBS37JTQWW4CFCFL6R5WUA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421830327 (602h22m53.305243957s) [noise=true , address=45.91.116.9:4261 , piece_id=RJPIMYLOKVTWT7OAPDSLZRDKAS6S24MDJJEDO7R6T6KES6QOTKJQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422004819 (602h22m53.305418454s) [seg_pos={0 0}, upload=481619]: Done begin upload segment. Jul 18 11:33:53.422067705 (602h22m53.305481341s) [noise=true , address=45.91.116.13:4154 , piece_id=3DWHEX64WIUEFNTA7OQSI3N246VV5PL656S4BBZLKZRRFSYYPBSA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422255208 (602h22m53.305668836s) [noise=true , address=77.90.31.14:4662 , piece_id=3QLWYPH4XRRGSGS2TWRLD33ZYGAWGQTUOHVRNARWKSFLAXDD5V6A, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422510516 (602h22m53.305924152s) [noise=true , address=45.91.116.5:4298 , piece_id=MJCI34UOQBT7QVK5DJU35742ISYLWFUXYZKNZB2JDY2VVTVRNSKA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422891686 (602h22m53.306305323s) [noise=true , address=104.36.82.14:4223 , piece_id=HTLVOCDVQC25FPTK47EPLVESUQF4SQBARDGX34XF4H44S2KHR4KA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.423286438 (602h22m53.306700076s) [noise=true , address=77.90.31.16:4733 , piece_id=7GKK3MIAQZE55KUZ7CISUAKVOIY253FA5X3IPOEB46KIGZYHMQGQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.423650064 (602h22m53.307063701s) [noise=true , address=77.90.31.11:4555 , piece_id=36G5A5ZFW63MBSXUI5PUMXRWQWMIRUYB57WKV5MSFHIPOTWNGUQA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424001413 (602h22m53.30741505s) [noise=true , address=45.91.116.8:4047 , piece_id=TH7446RXMI3N2KVGWO4RFEJRJT7VNQK3BCOV4LX3NPOKFUPDLPAQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424340658 (602h22m53.307754287s) [noise=true , address=104.36.82.8:4971 , piece_id=GDLNL7U7CWXP32IIHOCDOCOFU5LFCQIOUE7Y5JR27BYGSGPBH5DQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424783800 (602h22m53.308197433s) [noise=true , address=104.36.82.4:4801 , piece_id=DUDK4ODWMXKWOX3T44XQANH7K6IDJY5XUYAZAD6URYVQVD2EGGNA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:54.037250564 (602h22m53.920664177s) [upload=481619]: Next returned nil segment Jul 18 11:33:54.037268249 (602h22m53.920681862s) [upload=481619]: Waiting for error group managing segments... Jul 18 11:33:54.174223271 (602h22m54.057636898s) [noise=true , address=45.91.116.5:4299 , piece_id=6MPSMO2NGZF7KUKW62F6DV6MB7EUCUSBS37JTQWW4CFCFL6R5WUA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.175017283 (602h22m54.058430971s) [noise=true , address=45.91.116.13:4154 , piece_id=3DWHEX64WIUEFNTA7OQSI3N246VV5PL656S4BBZLKZRRFSYYPBSA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.186693354 (602h22m54.070106972s) [noise=true , address=77.90.31.10:4519 , piece_id=FZLX3JO2323KSQ2LRF7ZQIJ54SKUQOFADMJ4RV4MLDCR7LAKYLGA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.194210810 (602h22m54.077624423s) [noise=true , address=45.91.116.12:4227 , piece_id=YZHWNLTXU4C7U6BD4ZMOASYHRKWOJX4SHTYSZCBFSCUCSIQGWKOA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.194454877 (602h22m54.077868504s) [noise=true , address=45.91.116.9:4261 , piece_id=RJPIMYLOKVTWT7OAPDSLZRDKAS6S24MDJJEDO7R6T6KES6QOTKJQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.198427713 (602h22m54.08184133s) [noise=true , address=45.91.116.5:4298 , piece_id=MJCI34UOQBT7QVK5DJU35742ISYLWFUXYZKNZB2JDY2VVTVRNSKA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.198472131 (602h22m54.081885751s) [noise=true , address=45.91.116.8:4047 , piece_id=TH7446RXMI3N2KVGWO4RFEJRJT7VNQK3BCOV4LX3NPOKFUPDLPAQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.207083885 (602h22m54.090497512s) [noise=true , address=104.36.82.4:4801 , piece_id=DUDK4ODWMXKWOX3T44XQANH7K6IDJY5XUYAZAD6URYVQVD2EGGNA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.208816335 (602h22m54.09222996s) [noise=true , address=104.36.82.8:4971 , piece_id=GDLNL7U7CWXP32IIHOCDOCOFU5LFCQIOUE7Y5JR27BYGSGPBH5DQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.336523319 (602h22m54.219936957s) [noise=true , address=104.36.82.14:4223 , piece_id=HTLVOCDVQC25FPTK47EPLVESUQF4SQBARDGX34XF4H44S2KHR4KA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:55.287929075 (602h22m55.171342693s) [noise=true , address=45.91.116.10:4082 , piece_id=MTW24CAAS7GMDLOJPXNHI6GEAZMREH5EXQ4DSZGAOJFX6PMJRYLQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:34:23.031611818 (602h23m22.915025445s) [noise=true , address=77.90.31.18:4805 , piece_id=MEPMORRPK3YN2CH6I5CCXPWVKS2J2FLFWBZYXZPGPKTWLMVWO7UA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: rpc: dial tcp 77.90.31.18:4805: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031747683 (602h23m22.915161316s) [noise=true , address=77.90.31.11:4555 , piece_id=36G5A5ZFW63MBSXUI5PUMXRWQWMIRUYB57WKV5MSFHIPOTWNGUQA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 139.59.144.22:43832->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031825318 (602h23m22.915238945s) [noise=true , address=77.90.31.16:4733 , piece_id=7GKK3MIAQZE55KUZ7CISUAKVOIY253FA5X3IPOEB46KIGZYHMQGQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:37494->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031906140 (602h23m22.915319768s) [noise=true , address=77.90.31.15:4699 , piece_id=YJBX34SLFMFNBBFJA3G2KDKB4HGIWY743XTQ5OHVFE5RJUDKYLXA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:45528->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031982671 (602h23m22.9153963s) [noise=true , address=77.90.31.14:4662 , piece_id=3QLWYPH4XRRGSGS2TWRLD33ZYGAWGQTUOHVRNARWKSFLAXDD5V6A, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:44356->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.032005226 (602h23m22.915418863s) [seg_pos={0 0}, upload=481619]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 11:34:23.158376157 (602h23m23.041789785s) [upload=481619]: Done uploading segments... Jul 18 11:34:23.158427828 (602h23m23.041841454s) [upload=481619]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.432800775s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.744983739s FAIL (2025-07-18 11:35:17 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.745018229s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 1.117561778s FAIL (2025-07-18 11:35:05 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.11762961s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [............X.....X.X..........X...............X..........X.] duration: 2.37623859s FAIL (2025-07-18 11:34:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== , upload=481608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:32:54.154838589 (602h21m54.038252206s) [upload=481608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:32:54.154843826 (602h21m54.038257453s) [upload=481608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:32:54.305725283 (602h21m54.189138898s) [upload=481608]: Done uploading segments... Jul 18 11:32:54.305745379 (602h21m54.189159007s) [upload=481608]: Upload finished. err: <nil> Jul 18 11:33:53.306124309 (602h22m53.189537928s) [upload=481619]: Starting upload Jul 18 11:33:53.306205552 (602h22m53.18961919s) [upload=481619]: Done starting upload Jul 18 11:33:53.306275544 (602h22m53.189689161s) [upload=481619]: Uploading segments... Jul 18 11:33:53.306475959 (602h22m53.189889586s) [upload=481619]: Got next segment. Inline: false Jul 18 11:33:53.306513703 (602h22m53.18992732s) [upload=481619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:33:53.306519968 (602h22m53.189933586s) [upload=481619]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:33:53.419183467 (602h22m53.302597096s) [seg_pos={0 0}, upload=481619]: Begin upload segment... Jul 18 11:33:53.419316113 (602h22m53.302729751s) [noise=true , address=45.91.116.12:4227 , piece_id=YZHWNLTXU4C7U6BD4ZMOASYHRKWOJX4SHTYSZCBFSCUCSIQGWKOA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.419850102 (602h22m53.303263734s) [noise=true , address=77.90.31.15:4699 , piece_id=YJBX34SLFMFNBBFJA3G2KDKB4HGIWY743XTQ5OHVFE5RJUDKYLXA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.420226364 (602h22m53.30364s) [noise=true , address=45.91.116.10:4082 , piece_id=MTW24CAAS7GMDLOJPXNHI6GEAZMREH5EXQ4DSZGAOJFX6PMJRYLQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.420790778 (602h22m53.304204413s) [noise=true , address=77.90.31.10:4519 , piece_id=FZLX3JO2323KSQ2LRF7ZQIJ54SKUQOFADMJ4RV4MLDCR7LAKYLGA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421151603 (602h22m53.304565232s) [noise=true , address=77.90.31.18:4805 , piece_id=MEPMORRPK3YN2CH6I5CCXPWVKS2J2FLFWBZYXZPGPKTWLMVWO7UA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421473878 (602h22m53.304887505s) [noise=true , address=45.91.116.5:4299 , piece_id=6MPSMO2NGZF7KUKW62F6DV6MB7EUCUSBS37JTQWW4CFCFL6R5WUA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421830327 (602h22m53.305243957s) [noise=true , address=45.91.116.9:4261 , piece_id=RJPIMYLOKVTWT7OAPDSLZRDKAS6S24MDJJEDO7R6T6KES6QOTKJQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422004819 (602h22m53.305418454s) [seg_pos={0 0}, upload=481619]: Done begin upload segment. Jul 18 11:33:53.422067705 (602h22m53.305481341s) [noise=true , address=45.91.116.13:4154 , piece_id=3DWHEX64WIUEFNTA7OQSI3N246VV5PL656S4BBZLKZRRFSYYPBSA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422255208 (602h22m53.305668836s) [noise=true , address=77.90.31.14:4662 , piece_id=3QLWYPH4XRRGSGS2TWRLD33ZYGAWGQTUOHVRNARWKSFLAXDD5V6A, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422510516 (602h22m53.305924152s) [noise=true , address=45.91.116.5:4298 , piece_id=MJCI34UOQBT7QVK5DJU35742ISYLWFUXYZKNZB2JDY2VVTVRNSKA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422891686 (602h22m53.306305323s) [noise=true , address=104.36.82.14:4223 , piece_id=HTLVOCDVQC25FPTK47EPLVESUQF4SQBARDGX34XF4H44S2KHR4KA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.423286438 (602h22m53.306700076s) [noise=true , address=77.90.31.16:4733 , piece_id=7GKK3MIAQZE55KUZ7CISUAKVOIY253FA5X3IPOEB46KIGZYHMQGQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.423650064 (602h22m53.307063701s) [noise=true , address=77.90.31.11:4555 , piece_id=36G5A5ZFW63MBSXUI5PUMXRWQWMIRUYB57WKV5MSFHIPOTWNGUQA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424001413 (602h22m53.30741505s) [noise=true , address=45.91.116.8:4047 , piece_id=TH7446RXMI3N2KVGWO4RFEJRJT7VNQK3BCOV4LX3NPOKFUPDLPAQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424340658 (602h22m53.307754287s) [noise=true , address=104.36.82.8:4971 , piece_id=GDLNL7U7CWXP32IIHOCDOCOFU5LFCQIOUE7Y5JR27BYGSGPBH5DQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424783800 (602h22m53.308197433s) [noise=true , address=104.36.82.4:4801 , piece_id=DUDK4ODWMXKWOX3T44XQANH7K6IDJY5XUYAZAD6URYVQVD2EGGNA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:54.037250564 (602h22m53.920664177s) [upload=481619]: Next returned nil segment Jul 18 11:33:54.037268249 (602h22m53.920681862s) [upload=481619]: Waiting for error group managing segments... Jul 18 11:33:54.174223271 (602h22m54.057636898s) [noise=true , address=45.91.116.5:4299 , piece_id=6MPSMO2NGZF7KUKW62F6DV6MB7EUCUSBS37JTQWW4CFCFL6R5WUA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.175017283 (602h22m54.058430971s) [noise=true , address=45.91.116.13:4154 , piece_id=3DWHEX64WIUEFNTA7OQSI3N246VV5PL656S4BBZLKZRRFSYYPBSA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.186693354 (602h22m54.070106972s) [noise=true , address=77.90.31.10:4519 , piece_id=FZLX3JO2323KSQ2LRF7ZQIJ54SKUQOFADMJ4RV4MLDCR7LAKYLGA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.194210810 (602h22m54.077624423s) [noise=true , address=45.91.116.12:4227 , piece_id=YZHWNLTXU4C7U6BD4ZMOASYHRKWOJX4SHTYSZCBFSCUCSIQGWKOA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.194454877 (602h22m54.077868504s) [noise=true , address=45.91.116.9:4261 , piece_id=RJPIMYLOKVTWT7OAPDSLZRDKAS6S24MDJJEDO7R6T6KES6QOTKJQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.198427713 (602h22m54.08184133s) [noise=true , address=45.91.116.5:4298 , piece_id=MJCI34UOQBT7QVK5DJU35742ISYLWFUXYZKNZB2JDY2VVTVRNSKA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.198472131 (602h22m54.081885751s) [noise=true , address=45.91.116.8:4047 , piece_id=TH7446RXMI3N2KVGWO4RFEJRJT7VNQK3BCOV4LX3NPOKFUPDLPAQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.207083885 (602h22m54.090497512s) [noise=true , address=104.36.82.4:4801 , piece_id=DUDK4ODWMXKWOX3T44XQANH7K6IDJY5XUYAZAD6URYVQVD2EGGNA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.208816335 (602h22m54.09222996s) [noise=true , address=104.36.82.8:4971 , piece_id=GDLNL7U7CWXP32IIHOCDOCOFU5LFCQIOUE7Y5JR27BYGSGPBH5DQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.336523319 (602h22m54.219936957s) [noise=true , address=104.36.82.14:4223 , piece_id=HTLVOCDVQC25FPTK47EPLVESUQF4SQBARDGX34XF4H44S2KHR4KA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:55.287929075 (602h22m55.171342693s) [noise=true , address=45.91.116.10:4082 , piece_id=MTW24CAAS7GMDLOJPXNHI6GEAZMREH5EXQ4DSZGAOJFX6PMJRYLQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:34:23.031611818 (602h23m22.915025445s) [noise=true , address=77.90.31.18:4805 , piece_id=MEPMORRPK3YN2CH6I5CCXPWVKS2J2FLFWBZYXZPGPKTWLMVWO7UA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: rpc: dial tcp 77.90.31.18:4805: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031747683 (602h23m22.915161316s) [noise=true , address=77.90.31.11:4555 , piece_id=36G5A5ZFW63MBSXUI5PUMXRWQWMIRUYB57WKV5MSFHIPOTWNGUQA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 139.59.144.22:43832->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031825318 (602h23m22.915238945s) [noise=true , address=77.90.31.16:4733 , piece_id=7GKK3MIAQZE55KUZ7CISUAKVOIY253FA5X3IPOEB46KIGZYHMQGQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:37494->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031906140 (602h23m22.915319768s) [noise=true , address=77.90.31.15:4699 , piece_id=YJBX34SLFMFNBBFJA3G2KDKB4HGIWY743XTQ5OHVFE5RJUDKYLXA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:45528->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031982671 (602h23m22.9153963s) [noise=true , address=77.90.31.14:4662 , piece_id=3QLWYPH4XRRGSGS2TWRLD33ZYGAWGQTUOHVRNARWKSFLAXDD5V6A, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:44356->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.032005226 (602h23m22.915418863s) [seg_pos={0 0}, upload=481619]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 11:34:23.158376157 (602h23m23.041789785s) [upload=481619]: Done uploading segments... Jul 18 11:34:23.158427828 (602h23m23.041841454s) [upload=481619]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.376262903s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.............X.....X.X..........X...............X..........X] duration: 30.129251463s FAIL (2025-07-18 11:34:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== , upload=481608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:32:54.154838589 (602h21m54.038252206s) [upload=481608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:32:54.154843826 (602h21m54.038257453s) [upload=481608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:32:54.305725283 (602h21m54.189138898s) [upload=481608]: Done uploading segments... Jul 18 11:32:54.305745379 (602h21m54.189159007s) [upload=481608]: Upload finished. err: <nil> Jul 18 11:33:53.306124309 (602h22m53.189537928s) [upload=481619]: Starting upload Jul 18 11:33:53.306205552 (602h22m53.18961919s) [upload=481619]: Done starting upload Jul 18 11:33:53.306275544 (602h22m53.189689161s) [upload=481619]: Uploading segments... Jul 18 11:33:53.306475959 (602h22m53.189889586s) [upload=481619]: Got next segment. Inline: false Jul 18 11:33:53.306513703 (602h22m53.18992732s) [upload=481619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:33:53.306519968 (602h22m53.189933586s) [upload=481619]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:33:53.419183467 (602h22m53.302597096s) [seg_pos={0 0}, upload=481619]: Begin upload segment... Jul 18 11:33:53.419316113 (602h22m53.302729751s) [noise=true , address=45.91.116.12:4227 , piece_id=YZHWNLTXU4C7U6BD4ZMOASYHRKWOJX4SHTYSZCBFSCUCSIQGWKOA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.419850102 (602h22m53.303263734s) [noise=true , address=77.90.31.15:4699 , piece_id=YJBX34SLFMFNBBFJA3G2KDKB4HGIWY743XTQ5OHVFE5RJUDKYLXA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.420226364 (602h22m53.30364s) [noise=true , address=45.91.116.10:4082 , piece_id=MTW24CAAS7GMDLOJPXNHI6GEAZMREH5EXQ4DSZGAOJFX6PMJRYLQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.420790778 (602h22m53.304204413s) [noise=true , address=77.90.31.10:4519 , piece_id=FZLX3JO2323KSQ2LRF7ZQIJ54SKUQOFADMJ4RV4MLDCR7LAKYLGA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421151603 (602h22m53.304565232s) [noise=true , address=77.90.31.18:4805 , piece_id=MEPMORRPK3YN2CH6I5CCXPWVKS2J2FLFWBZYXZPGPKTWLMVWO7UA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421473878 (602h22m53.304887505s) [noise=true , address=45.91.116.5:4299 , piece_id=6MPSMO2NGZF7KUKW62F6DV6MB7EUCUSBS37JTQWW4CFCFL6R5WUA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.421830327 (602h22m53.305243957s) [noise=true , address=45.91.116.9:4261 , piece_id=RJPIMYLOKVTWT7OAPDSLZRDKAS6S24MDJJEDO7R6T6KES6QOTKJQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422004819 (602h22m53.305418454s) [seg_pos={0 0}, upload=481619]: Done begin upload segment. Jul 18 11:33:53.422067705 (602h22m53.305481341s) [noise=true , address=45.91.116.13:4154 , piece_id=3DWHEX64WIUEFNTA7OQSI3N246VV5PL656S4BBZLKZRRFSYYPBSA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422255208 (602h22m53.305668836s) [noise=true , address=77.90.31.14:4662 , piece_id=3QLWYPH4XRRGSGS2TWRLD33ZYGAWGQTUOHVRNARWKSFLAXDD5V6A, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422510516 (602h22m53.305924152s) [noise=true , address=45.91.116.5:4298 , piece_id=MJCI34UOQBT7QVK5DJU35742ISYLWFUXYZKNZB2JDY2VVTVRNSKA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.422891686 (602h22m53.306305323s) [noise=true , address=104.36.82.14:4223 , piece_id=HTLVOCDVQC25FPTK47EPLVESUQF4SQBARDGX34XF4H44S2KHR4KA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.423286438 (602h22m53.306700076s) [noise=true , address=77.90.31.16:4733 , piece_id=7GKK3MIAQZE55KUZ7CISUAKVOIY253FA5X3IPOEB46KIGZYHMQGQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.423650064 (602h22m53.307063701s) [noise=true , address=77.90.31.11:4555 , piece_id=36G5A5ZFW63MBSXUI5PUMXRWQWMIRUYB57WKV5MSFHIPOTWNGUQA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424001413 (602h22m53.30741505s) [noise=true , address=45.91.116.8:4047 , piece_id=TH7446RXMI3N2KVGWO4RFEJRJT7VNQK3BCOV4LX3NPOKFUPDLPAQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424340658 (602h22m53.307754287s) [noise=true , address=104.36.82.8:4971 , piece_id=GDLNL7U7CWXP32IIHOCDOCOFU5LFCQIOUE7Y5JR27BYGSGPBH5DQ, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:53.424783800 (602h22m53.308197433s) [noise=true , address=104.36.82.4:4801 , piece_id=DUDK4ODWMXKWOX3T44XQANH7K6IDJY5XUYAZAD6URYVQVD2EGGNA, seg_pos={0 0}, upload=481619]: Uploading piece... Jul 18 11:33:54.037250564 (602h22m53.920664177s) [upload=481619]: Next returned nil segment Jul 18 11:33:54.037268249 (602h22m53.920681862s) [upload=481619]: Waiting for error group managing segments... Jul 18 11:33:54.174223271 (602h22m54.057636898s) [noise=true , address=45.91.116.5:4299 , piece_id=6MPSMO2NGZF7KUKW62F6DV6MB7EUCUSBS37JTQWW4CFCFL6R5WUA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.175017283 (602h22m54.058430971s) [noise=true , address=45.91.116.13:4154 , piece_id=3DWHEX64WIUEFNTA7OQSI3N246VV5PL656S4BBZLKZRRFSYYPBSA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.186693354 (602h22m54.070106972s) [noise=true , address=77.90.31.10:4519 , piece_id=FZLX3JO2323KSQ2LRF7ZQIJ54SKUQOFADMJ4RV4MLDCR7LAKYLGA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.194210810 (602h22m54.077624423s) [noise=true , address=45.91.116.12:4227 , piece_id=YZHWNLTXU4C7U6BD4ZMOASYHRKWOJX4SHTYSZCBFSCUCSIQGWKOA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.194454877 (602h22m54.077868504s) [noise=true , address=45.91.116.9:4261 , piece_id=RJPIMYLOKVTWT7OAPDSLZRDKAS6S24MDJJEDO7R6T6KES6QOTKJQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.198427713 (602h22m54.08184133s) [noise=true , address=45.91.116.5:4298 , piece_id=MJCI34UOQBT7QVK5DJU35742ISYLWFUXYZKNZB2JDY2VVTVRNSKA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.198472131 (602h22m54.081885751s) [noise=true , address=45.91.116.8:4047 , piece_id=TH7446RXMI3N2KVGWO4RFEJRJT7VNQK3BCOV4LX3NPOKFUPDLPAQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.207083885 (602h22m54.090497512s) [noise=true , address=104.36.82.4:4801 , piece_id=DUDK4ODWMXKWOX3T44XQANH7K6IDJY5XUYAZAD6URYVQVD2EGGNA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.208816335 (602h22m54.09222996s) [noise=true , address=104.36.82.8:4971 , piece_id=GDLNL7U7CWXP32IIHOCDOCOFU5LFCQIOUE7Y5JR27BYGSGPBH5DQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:54.336523319 (602h22m54.219936957s) [noise=true , address=104.36.82.14:4223 , piece_id=HTLVOCDVQC25FPTK47EPLVESUQF4SQBARDGX34XF4H44S2KHR4KA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:33:55.287929075 (602h22m55.171342693s) [noise=true , address=45.91.116.10:4082 , piece_id=MTW24CAAS7GMDLOJPXNHI6GEAZMREH5EXQ4DSZGAOJFX6PMJRYLQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: <nil> Jul 18 11:34:23.031611818 (602h23m22.915025445s) [noise=true , address=77.90.31.18:4805 , piece_id=MEPMORRPK3YN2CH6I5CCXPWVKS2J2FLFWBZYXZPGPKTWLMVWO7UA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: rpc: dial tcp 77.90.31.18:4805: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031747683 (602h23m22.915161316s) [noise=true , address=77.90.31.11:4555 , piece_id=36G5A5ZFW63MBSXUI5PUMXRWQWMIRUYB57WKV5MSFHIPOTWNGUQA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 139.59.144.22:43832->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031825318 (602h23m22.915238945s) [noise=true , address=77.90.31.16:4733 , piece_id=7GKK3MIAQZE55KUZ7CISUAKVOIY253FA5X3IPOEB46KIGZYHMQGQ, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:37494->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031906140 (602h23m22.915319768s) [noise=true , address=77.90.31.15:4699 , piece_id=YJBX34SLFMFNBBFJA3G2KDKB4HGIWY743XTQ5OHVFE5RJUDKYLXA, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:45528->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.031982671 (602h23m22.9153963s) [noise=true , address=77.90.31.14:4662 , piece_id=3QLWYPH4XRRGSGS2TWRLD33ZYGAWGQTUOHVRNARWKSFLAXDD5V6A, seg_pos={0 0}, upload=481619]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:44356->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:34:23.032005226 (602h23m22.915418863s) [seg_pos={0 0}, upload=481619]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 11:34:23.158376157 (602h23m23.041789785s) [upload=481619]: Done uploading segments... Jul 18 11:34:23.158427828 (602h23m23.041841454s) [upload=481619]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.129277836s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.606906169s FAIL (2025-07-18 11:34:11 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.606937355s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 1.262189486s FAIL (2025-07-18 11:34:05 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.262227461s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 1.09125904s FAIL (2025-07-18 11:33:26 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.091281542s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 3.032743886s FAIL (2025-07-18 11:33:09 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 3.032843394s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..............X.....X.X..........X...............X..........] duration: 2.338732033s FAIL (2025-07-18 11:32:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 139.59.144.22:52386->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:25:03.756616000 (602h14m3.640029628s) [seg_pos={0 0}, upload=481509]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:25:03.756750542 (602h14m3.640164173s) [upload=481509]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:25:03.756757773 (602h14m3.640171571s) [upload=481509]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:25:03.918610530 (602h14m3.802024164s) [upload=481509]: Done uploading segments... Jul 18 11:25:03.918632896 (602h14m3.802046512s) [upload=481509]: Upload finished. err: <nil> Jul 18 11:25:56.053888090 (602h14m55.937301716s) [upload=481521]: Starting upload Jul 18 11:25:56.053994103 (602h14m55.937407737s) [upload=481521]: Done starting upload Jul 18 11:25:56.054129122 (602h14m55.937542749s) [upload=481521]: Uploading segments... Jul 18 11:25:56.054306891 (602h14m55.937720517s) [upload=481521]: Got next segment. Inline: false Jul 18 11:25:56.054364770 (602h14m55.937778397s) [upload=481521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:25:56.054371001 (602h14m55.937784628s) [upload=481521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:25:56.170318697 (602h14m56.053732323s) [seg_pos={0 0}, upload=481521]: Begin upload segment... Jul 18 11:25:56.170422900 (602h14m56.053836526s) [seg_pos={0 0}, upload=481521]: Done begin upload segment. Jul 18 11:25:56.170464681 (602h14m56.053878314s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170521320 (602h14m56.053934956s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170853558 (602h14m56.054267185s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170909345 (602h14m56.054322977s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171219719 (602h14m56.054633352s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171337279 (602h14m56.054750915s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171541852 (602h14m56.054955482s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171704517 (602h14m56.055118151s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171920013 (602h14m56.055333643s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172277758 (602h14m56.055691391s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172326432 (602h14m56.055740064s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172568781 (602h14m56.055982394s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172668061 (602h14m56.056081694s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172832081 (602h14m56.056245713s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173001844 (602h14m56.056415473s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173339770 (602h14m56.056753404s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.786495143 (602h14m56.669908759s) [upload=481521]: Next returned nil segment Jul 18 11:25:56.786512550 (602h14m56.669926162s) [upload=481521]: Waiting for error group managing segments... Jul 18 11:25:56.918094694 (602h14m56.80150833s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.918628075 (602h14m56.802041694s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.919912323 (602h14m56.803325943s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.920806034 (602h14m56.804219647s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.921420465 (602h14m56.804834084s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.923349719 (602h14m56.80676334s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.926920740 (602h14m56.810334354s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.930546114 (602h14m56.813959734s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.931434176 (602h14m56.814847796s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.945660373 (602h14m56.829074004s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.955452695 (602h14m56.838866315s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.957436329 (602h14m56.840849962s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:26:25.766597801 (602h15m25.65001142s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766639011 (602h15m25.650052626s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:53448->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766658832 (602h15m25.650072444s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766671419 (602h15m25.650085044s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:48546->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766698381 (602h15m25.650112007s) [seg_pos={0 0}, upload=481521]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 11:26:25.884294106 (602h15m25.76770774s) [upload=481521]: Done uploading segments... Jul 18 11:26:25.884340118 (602h15m25.767753759s) [upload=481521]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.338760021s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 1.455978811s FAIL (2025-07-18 11:32:48 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.456004088s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.638971525s FAIL (2025-07-18 11:32:31 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.639019252s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 3.037264924s FAIL (2025-07-18 11:32:02 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 3.037304854s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...............X.....X.X..........X...............X.........] duration: 2.354279866s FAIL (2025-07-18 11:31:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 139.59.144.22:52386->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:25:03.756616000 (602h14m3.640029628s) [seg_pos={0 0}, upload=481509]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:25:03.756750542 (602h14m3.640164173s) [upload=481509]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:25:03.756757773 (602h14m3.640171571s) [upload=481509]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:25:03.918610530 (602h14m3.802024164s) [upload=481509]: Done uploading segments... Jul 18 11:25:03.918632896 (602h14m3.802046512s) [upload=481509]: Upload finished. err: <nil> Jul 18 11:25:56.053888090 (602h14m55.937301716s) [upload=481521]: Starting upload Jul 18 11:25:56.053994103 (602h14m55.937407737s) [upload=481521]: Done starting upload Jul 18 11:25:56.054129122 (602h14m55.937542749s) [upload=481521]: Uploading segments... Jul 18 11:25:56.054306891 (602h14m55.937720517s) [upload=481521]: Got next segment. Inline: false Jul 18 11:25:56.054364770 (602h14m55.937778397s) [upload=481521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:25:56.054371001 (602h14m55.937784628s) [upload=481521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:25:56.170318697 (602h14m56.053732323s) [seg_pos={0 0}, upload=481521]: Begin upload segment... Jul 18 11:25:56.170422900 (602h14m56.053836526s) [seg_pos={0 0}, upload=481521]: Done begin upload segment. Jul 18 11:25:56.170464681 (602h14m56.053878314s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170521320 (602h14m56.053934956s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170853558 (602h14m56.054267185s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170909345 (602h14m56.054322977s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171219719 (602h14m56.054633352s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171337279 (602h14m56.054750915s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171541852 (602h14m56.054955482s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171704517 (602h14m56.055118151s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171920013 (602h14m56.055333643s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172277758 (602h14m56.055691391s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172326432 (602h14m56.055740064s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172568781 (602h14m56.055982394s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172668061 (602h14m56.056081694s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172832081 (602h14m56.056245713s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173001844 (602h14m56.056415473s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173339770 (602h14m56.056753404s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.786495143 (602h14m56.669908759s) [upload=481521]: Next returned nil segment Jul 18 11:25:56.786512550 (602h14m56.669926162s) [upload=481521]: Waiting for error group managing segments... Jul 18 11:25:56.918094694 (602h14m56.80150833s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.918628075 (602h14m56.802041694s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.919912323 (602h14m56.803325943s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.920806034 (602h14m56.804219647s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.921420465 (602h14m56.804834084s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.923349719 (602h14m56.80676334s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.926920740 (602h14m56.810334354s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.930546114 (602h14m56.813959734s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.931434176 (602h14m56.814847796s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.945660373 (602h14m56.829074004s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.955452695 (602h14m56.838866315s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.957436329 (602h14m56.840849962s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:26:25.766597801 (602h15m25.65001142s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766639011 (602h15m25.650052626s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:53448->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766658832 (602h15m25.650072444s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766671419 (602h15m25.650085044s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:48546->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766698381 (602h15m25.650112007s) [seg_pos={0 0}, upload=481521]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 11:26:25.884294106 (602h15m25.76770774s) [upload=481521]: Done uploading segments... Jul 18 11:26:25.884340118 (602h15m25.767753759s) [upload=481521]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.354306203s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 1.080467369s FAIL (2025-07-18 11:31:38 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.080583829s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [................X.....X.X..........X...............X........] duration: 2.414503174s FAIL (2025-07-18 11:31:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 139.59.144.22:52386->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:25:03.756616000 (602h14m3.640029628s) [seg_pos={0 0}, upload=481509]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:25:03.756750542 (602h14m3.640164173s) [upload=481509]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:25:03.756757773 (602h14m3.640171571s) [upload=481509]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:25:03.918610530 (602h14m3.802024164s) [upload=481509]: Done uploading segments... Jul 18 11:25:03.918632896 (602h14m3.802046512s) [upload=481509]: Upload finished. err: <nil> Jul 18 11:25:56.053888090 (602h14m55.937301716s) [upload=481521]: Starting upload Jul 18 11:25:56.053994103 (602h14m55.937407737s) [upload=481521]: Done starting upload Jul 18 11:25:56.054129122 (602h14m55.937542749s) [upload=481521]: Uploading segments... Jul 18 11:25:56.054306891 (602h14m55.937720517s) [upload=481521]: Got next segment. Inline: false Jul 18 11:25:56.054364770 (602h14m55.937778397s) [upload=481521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:25:56.054371001 (602h14m55.937784628s) [upload=481521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:25:56.170318697 (602h14m56.053732323s) [seg_pos={0 0}, upload=481521]: Begin upload segment... Jul 18 11:25:56.170422900 (602h14m56.053836526s) [seg_pos={0 0}, upload=481521]: Done begin upload segment. Jul 18 11:25:56.170464681 (602h14m56.053878314s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170521320 (602h14m56.053934956s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170853558 (602h14m56.054267185s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170909345 (602h14m56.054322977s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171219719 (602h14m56.054633352s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171337279 (602h14m56.054750915s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171541852 (602h14m56.054955482s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171704517 (602h14m56.055118151s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171920013 (602h14m56.055333643s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172277758 (602h14m56.055691391s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172326432 (602h14m56.055740064s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172568781 (602h14m56.055982394s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172668061 (602h14m56.056081694s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172832081 (602h14m56.056245713s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173001844 (602h14m56.056415473s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173339770 (602h14m56.056753404s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.786495143 (602h14m56.669908759s) [upload=481521]: Next returned nil segment Jul 18 11:25:56.786512550 (602h14m56.669926162s) [upload=481521]: Waiting for error group managing segments... Jul 18 11:25:56.918094694 (602h14m56.80150833s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.918628075 (602h14m56.802041694s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.919912323 (602h14m56.803325943s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.920806034 (602h14m56.804219647s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.921420465 (602h14m56.804834084s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.923349719 (602h14m56.80676334s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.926920740 (602h14m56.810334354s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.930546114 (602h14m56.813959734s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.931434176 (602h14m56.814847796s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.945660373 (602h14m56.829074004s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.955452695 (602h14m56.838866315s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.957436329 (602h14m56.840849962s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:26:25.766597801 (602h15m25.65001142s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766639011 (602h15m25.650052626s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:53448->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766658832 (602h15m25.650072444s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766671419 (602h15m25.650085044s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:48546->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766698381 (602h15m25.650112007s) [seg_pos={0 0}, upload=481521]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 11:26:25.884294106 (602h15m25.76770774s) [upload=481521]: Done uploading segments... Jul 18 11:26:25.884340118 (602h15m25.767753759s) [upload=481521]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.41453092s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.706877195s FAIL (2025-07-18 11:31:17 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.706914201s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.................X.....X.X..........X...............X.......] duration: 2.471085818s FAIL (2025-07-18 11:31:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 139.59.144.22:52386->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:25:03.756616000 (602h14m3.640029628s) [seg_pos={0 0}, upload=481509]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:25:03.756750542 (602h14m3.640164173s) [upload=481509]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:25:03.756757773 (602h14m3.640171571s) [upload=481509]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:25:03.918610530 (602h14m3.802024164s) [upload=481509]: Done uploading segments... Jul 18 11:25:03.918632896 (602h14m3.802046512s) [upload=481509]: Upload finished. err: <nil> Jul 18 11:25:56.053888090 (602h14m55.937301716s) [upload=481521]: Starting upload Jul 18 11:25:56.053994103 (602h14m55.937407737s) [upload=481521]: Done starting upload Jul 18 11:25:56.054129122 (602h14m55.937542749s) [upload=481521]: Uploading segments... Jul 18 11:25:56.054306891 (602h14m55.937720517s) [upload=481521]: Got next segment. Inline: false Jul 18 11:25:56.054364770 (602h14m55.937778397s) [upload=481521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:25:56.054371001 (602h14m55.937784628s) [upload=481521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:25:56.170318697 (602h14m56.053732323s) [seg_pos={0 0}, upload=481521]: Begin upload segment... Jul 18 11:25:56.170422900 (602h14m56.053836526s) [seg_pos={0 0}, upload=481521]: Done begin upload segment. Jul 18 11:25:56.170464681 (602h14m56.053878314s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170521320 (602h14m56.053934956s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170853558 (602h14m56.054267185s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170909345 (602h14m56.054322977s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171219719 (602h14m56.054633352s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171337279 (602h14m56.054750915s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171541852 (602h14m56.054955482s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171704517 (602h14m56.055118151s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171920013 (602h14m56.055333643s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172277758 (602h14m56.055691391s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172326432 (602h14m56.055740064s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172568781 (602h14m56.055982394s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172668061 (602h14m56.056081694s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172832081 (602h14m56.056245713s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173001844 (602h14m56.056415473s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173339770 (602h14m56.056753404s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.786495143 (602h14m56.669908759s) [upload=481521]: Next returned nil segment Jul 18 11:25:56.786512550 (602h14m56.669926162s) [upload=481521]: Waiting for error group managing segments... Jul 18 11:25:56.918094694 (602h14m56.80150833s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.918628075 (602h14m56.802041694s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.919912323 (602h14m56.803325943s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.920806034 (602h14m56.804219647s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.921420465 (602h14m56.804834084s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.923349719 (602h14m56.80676334s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.926920740 (602h14m56.810334354s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.930546114 (602h14m56.813959734s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.931434176 (602h14m56.814847796s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.945660373 (602h14m56.829074004s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.955452695 (602h14m56.838866315s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.957436329 (602h14m56.840849962s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:26:25.766597801 (602h15m25.65001142s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766639011 (602h15m25.650052626s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:53448->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766658832 (602h15m25.650072444s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766671419 (602h15m25.650085044s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:48546->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766698381 (602h15m25.650112007s) [seg_pos={0 0}, upload=481521]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 11:26:25.884294106 (602h15m25.76770774s) [upload=481521]: Done uploading segments... Jul 18 11:26:25.884340118 (602h15m25.767753759s) [upload=481521]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.471121483s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 1.118037063s FAIL (2025-07-18 11:30:55 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.118069871s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..................X.....X.X..........X...............X......] duration: 2.402050073s FAIL (2025-07-18 11:30:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 139.59.144.22:52386->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:25:03.756616000 (602h14m3.640029628s) [seg_pos={0 0}, upload=481509]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:25:03.756750542 (602h14m3.640164173s) [upload=481509]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:25:03.756757773 (602h14m3.640171571s) [upload=481509]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:25:03.918610530 (602h14m3.802024164s) [upload=481509]: Done uploading segments... Jul 18 11:25:03.918632896 (602h14m3.802046512s) [upload=481509]: Upload finished. err: <nil> Jul 18 11:25:56.053888090 (602h14m55.937301716s) [upload=481521]: Starting upload Jul 18 11:25:56.053994103 (602h14m55.937407737s) [upload=481521]: Done starting upload Jul 18 11:25:56.054129122 (602h14m55.937542749s) [upload=481521]: Uploading segments... Jul 18 11:25:56.054306891 (602h14m55.937720517s) [upload=481521]: Got next segment. Inline: false Jul 18 11:25:56.054364770 (602h14m55.937778397s) [upload=481521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:25:56.054371001 (602h14m55.937784628s) [upload=481521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:25:56.170318697 (602h14m56.053732323s) [seg_pos={0 0}, upload=481521]: Begin upload segment... Jul 18 11:25:56.170422900 (602h14m56.053836526s) [seg_pos={0 0}, upload=481521]: Done begin upload segment. Jul 18 11:25:56.170464681 (602h14m56.053878314s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170521320 (602h14m56.053934956s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170853558 (602h14m56.054267185s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170909345 (602h14m56.054322977s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171219719 (602h14m56.054633352s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171337279 (602h14m56.054750915s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171541852 (602h14m56.054955482s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171704517 (602h14m56.055118151s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171920013 (602h14m56.055333643s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172277758 (602h14m56.055691391s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172326432 (602h14m56.055740064s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172568781 (602h14m56.055982394s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172668061 (602h14m56.056081694s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172832081 (602h14m56.056245713s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173001844 (602h14m56.056415473s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173339770 (602h14m56.056753404s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.786495143 (602h14m56.669908759s) [upload=481521]: Next returned nil segment Jul 18 11:25:56.786512550 (602h14m56.669926162s) [upload=481521]: Waiting for error group managing segments... Jul 18 11:25:56.918094694 (602h14m56.80150833s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.918628075 (602h14m56.802041694s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.919912323 (602h14m56.803325943s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.920806034 (602h14m56.804219647s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.921420465 (602h14m56.804834084s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.923349719 (602h14m56.80676334s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.926920740 (602h14m56.810334354s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.930546114 (602h14m56.813959734s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.931434176 (602h14m56.814847796s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.945660373 (602h14m56.829074004s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.955452695 (602h14m56.838866315s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.957436329 (602h14m56.840849962s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:26:25.766597801 (602h15m25.65001142s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766639011 (602h15m25.650052626s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:53448->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766658832 (602h15m25.650072444s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766671419 (602h15m25.650085044s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:48546->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766698381 (602h15m25.650112007s) [seg_pos={0 0}, upload=481521]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 11:26:25.884294106 (602h15m25.76770774s) [upload=481521]: Done uploading segments... Jul 18 11:26:25.884340118 (602h15m25.767753759s) [upload=481521]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.40214517s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 1.01839374s FAIL (2025-07-18 11:30:05 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.018418097s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 3.370103342s FAIL (2025-07-18 11:30:03 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 3.370137161s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...................X.....X.X..........X...............X.....] duration: 2.728336439s FAIL (2025-07-18 11:29:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 139.59.144.22:52386->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:25:03.756616000 (602h14m3.640029628s) [seg_pos={0 0}, upload=481509]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:25:03.756750542 (602h14m3.640164173s) [upload=481509]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:25:03.756757773 (602h14m3.640171571s) [upload=481509]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:25:03.918610530 (602h14m3.802024164s) [upload=481509]: Done uploading segments... Jul 18 11:25:03.918632896 (602h14m3.802046512s) [upload=481509]: Upload finished. err: <nil> Jul 18 11:25:56.053888090 (602h14m55.937301716s) [upload=481521]: Starting upload Jul 18 11:25:56.053994103 (602h14m55.937407737s) [upload=481521]: Done starting upload Jul 18 11:25:56.054129122 (602h14m55.937542749s) [upload=481521]: Uploading segments... Jul 18 11:25:56.054306891 (602h14m55.937720517s) [upload=481521]: Got next segment. Inline: false Jul 18 11:25:56.054364770 (602h14m55.937778397s) [upload=481521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:25:56.054371001 (602h14m55.937784628s) [upload=481521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:25:56.170318697 (602h14m56.053732323s) [seg_pos={0 0}, upload=481521]: Begin upload segment... Jul 18 11:25:56.170422900 (602h14m56.053836526s) [seg_pos={0 0}, upload=481521]: Done begin upload segment. Jul 18 11:25:56.170464681 (602h14m56.053878314s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170521320 (602h14m56.053934956s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170853558 (602h14m56.054267185s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170909345 (602h14m56.054322977s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171219719 (602h14m56.054633352s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171337279 (602h14m56.054750915s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171541852 (602h14m56.054955482s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171704517 (602h14m56.055118151s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171920013 (602h14m56.055333643s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172277758 (602h14m56.055691391s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172326432 (602h14m56.055740064s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172568781 (602h14m56.055982394s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172668061 (602h14m56.056081694s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172832081 (602h14m56.056245713s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173001844 (602h14m56.056415473s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173339770 (602h14m56.056753404s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.786495143 (602h14m56.669908759s) [upload=481521]: Next returned nil segment Jul 18 11:25:56.786512550 (602h14m56.669926162s) [upload=481521]: Waiting for error group managing segments... Jul 18 11:25:56.918094694 (602h14m56.80150833s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.918628075 (602h14m56.802041694s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.919912323 (602h14m56.803325943s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.920806034 (602h14m56.804219647s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.921420465 (602h14m56.804834084s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.923349719 (602h14m56.80676334s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.926920740 (602h14m56.810334354s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.930546114 (602h14m56.813959734s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.931434176 (602h14m56.814847796s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.945660373 (602h14m56.829074004s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.955452695 (602h14m56.838866315s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.957436329 (602h14m56.840849962s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:26:25.766597801 (602h15m25.65001142s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766639011 (602h15m25.650052626s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:53448->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766658832 (602h15m25.650072444s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766671419 (602h15m25.650085044s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:48546->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766698381 (602h15m25.650112007s) [seg_pos={0 0}, upload=481521]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 11:26:25.884294106 (602h15m25.76770774s) [upload=481521]: Done uploading segments... Jul 18 11:26:25.884340118 (602h15m25.767753759s) [upload=481521]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.728362459s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 1.139113544s FAIL (2025-07-18 11:29:10 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.139139257s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 3.224040098s FAIL (2025-07-18 11:28:52 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 3.224067788s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 1.145782461s FAIL (2025-07-18 11:28:39 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.145809949s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.536767071s FAIL (2025-07-18 11:28:30 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.536795544s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [....................X.....X.X..........X...............X....] duration: 2.385721745s FAIL (2025-07-18 11:28:16 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 139.59.144.22:52386->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:25:03.756616000 (602h14m3.640029628s) [seg_pos={0 0}, upload=481509]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:25:03.756750542 (602h14m3.640164173s) [upload=481509]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:25:03.756757773 (602h14m3.640171571s) [upload=481509]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:25:03.918610530 (602h14m3.802024164s) [upload=481509]: Done uploading segments... Jul 18 11:25:03.918632896 (602h14m3.802046512s) [upload=481509]: Upload finished. err: <nil> Jul 18 11:25:56.053888090 (602h14m55.937301716s) [upload=481521]: Starting upload Jul 18 11:25:56.053994103 (602h14m55.937407737s) [upload=481521]: Done starting upload Jul 18 11:25:56.054129122 (602h14m55.937542749s) [upload=481521]: Uploading segments... Jul 18 11:25:56.054306891 (602h14m55.937720517s) [upload=481521]: Got next segment. Inline: false Jul 18 11:25:56.054364770 (602h14m55.937778397s) [upload=481521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:25:56.054371001 (602h14m55.937784628s) [upload=481521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:25:56.170318697 (602h14m56.053732323s) [seg_pos={0 0}, upload=481521]: Begin upload segment... Jul 18 11:25:56.170422900 (602h14m56.053836526s) [seg_pos={0 0}, upload=481521]: Done begin upload segment. Jul 18 11:25:56.170464681 (602h14m56.053878314s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170521320 (602h14m56.053934956s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170853558 (602h14m56.054267185s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170909345 (602h14m56.054322977s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171219719 (602h14m56.054633352s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171337279 (602h14m56.054750915s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171541852 (602h14m56.054955482s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171704517 (602h14m56.055118151s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171920013 (602h14m56.055333643s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172277758 (602h14m56.055691391s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172326432 (602h14m56.055740064s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172568781 (602h14m56.055982394s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172668061 (602h14m56.056081694s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172832081 (602h14m56.056245713s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173001844 (602h14m56.056415473s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173339770 (602h14m56.056753404s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.786495143 (602h14m56.669908759s) [upload=481521]: Next returned nil segment Jul 18 11:25:56.786512550 (602h14m56.669926162s) [upload=481521]: Waiting for error group managing segments... Jul 18 11:25:56.918094694 (602h14m56.80150833s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.918628075 (602h14m56.802041694s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.919912323 (602h14m56.803325943s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.920806034 (602h14m56.804219647s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.921420465 (602h14m56.804834084s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.923349719 (602h14m56.80676334s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.926920740 (602h14m56.810334354s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.930546114 (602h14m56.813959734s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.931434176 (602h14m56.814847796s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.945660373 (602h14m56.829074004s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.955452695 (602h14m56.838866315s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.957436329 (602h14m56.840849962s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:26:25.766597801 (602h15m25.65001142s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766639011 (602h15m25.650052626s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:53448->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766658832 (602h15m25.650072444s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766671419 (602h15m25.650085044s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:48546->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766698381 (602h15m25.650112007s) [seg_pos={0 0}, upload=481521]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 11:26:25.884294106 (602h15m25.76770774s) [upload=481521]: Done uploading segments... Jul 18 11:26:25.884340118 (602h15m25.767753759s) [upload=481521]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.385755079s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 1.018238007s FAIL (2025-07-18 11:28:02 UTC) orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded Duration: 1.018291531s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.609067623s FAIL (2025-07-18 11:27:49 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== HFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567267208 (601h46m53.450680834s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.567698610 (601h46m53.451112239s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.570842951 (601h46m53.454256585s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571148694 (601h46m53.454562307s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.571617590 (601h46m53.455031203s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572194976 (601h46m53.455608605s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.572657235 (601h46m53.45607085s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573103737 (601h46m53.456517363s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.573559498 (601h46m53.45697311s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.575230525 (601h46m53.458644152s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.576872259 (601h46m53.460285885s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.578624092 (601h46m53.462037718s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.580779720 (601h46m53.464193352s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.583021682 (601h46m53.466435301s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.584553497 (601h46m53.467967132s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.585102122 (601h46m53.468515755s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:53.586584733 (601h46m53.46999835s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Uploading piece... Jul 18 10:57:54.118272982 (601h46m54.001686609s) [upload=481170]: Next returned nil segment Jul 18 10:57:54.118290512 (601h46m54.001704132s) [upload=481170]: Waiting for error group managing segments... Jul 18 10:57:54.265678399 (601h46m54.149092018s) [noise=true , address=77.90.31.19:4849 , piece_id=2MY5DNKS7AAE3ME4SJ5XWXHGAICES3VH5SP4ADDJXEACDNCFUK7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.272591913 (601h46m54.156005531s) [noise=true , address=77.90.31.16:4760 , piece_id=SLUYYAF2AL4AA7UNV74LBEC6COPCNEDAU7Z6JJGP6NZQRA4QGGSA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.273166902 (601h46m54.156580518s) [noise=true , address=77.90.31.18:4821 , piece_id=ZT34UIRYSJR7W63UBZP5433JLLRI3LX2D7VYLDEJCSFU26VG4TGQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.275724342 (601h46m54.15913796s) [noise=true , address=45.91.116.5:4323 , piece_id=MBT6BQWV46PCBWY46DJF5WGQRMECQEMLXZNUIOB6HQ3KFD7ZRKIA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278917091 (601h46m54.162330707s) [noise=true , address=45.91.116.12:4259 , piece_id=4HL24ORSSTX5HQ6CA5HKDUF7GDK72ELCZIV44TR4J247FR7SH4RQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.278945616 (601h46m54.162359243s) [noise=true , address=45.91.116.14:4222 , piece_id=MML4PDHMCEIHAZVDC2DBP45HNBGFL3GKANZKTCWRRQSOXFLVGH7A, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.283715439 (601h46m54.167129051s) [noise=true , address=45.91.116.6:4998 , piece_id=K7R34IO3TUEZJWXXKYXMJNP6TU6UA4KXTFC43XQIQCHDSM6NYROQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.284438587 (601h46m54.167852208s) [noise=true , address=79.127.219.48:4104 , piece_id=DFY23S6CJXIV2TOK3DM7IIMNKVA5K4OONUMDKO7RZI3NJMXPAWYQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.294188062 (601h46m54.1776017s) [noise=true , address=104.36.82.50:4025 , piece_id=K6X7WBLMFJ2OHOGZU5KHUETOW6JF6MJKE7XT2NZ5NYATISG6ULSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.434300533 (601h46m54.317714163s) [noise=true , address=45.91.95.3:4022 , piece_id=STAZHFFZ2HSXVHPYQSXBJ2G3U3E5CC6ICEWRNJQ25W7LPBU7TNZA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.435093696 (601h46m54.318507314s) [noise=true , address=45.91.95.4:4002 , piece_id=TRN4JO4MDWIL7NLFRBGC2TKCVY2UFTQW2BRAZHIYVAO7JMIFMF4Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.436583511 (601h46m54.31999714s) [noise=true , address=45.91.95.4:4008 , piece_id=2YB4HUDGAOGDY4X4KQIJWX6CMNU3DOZ327MEFTRG5BDZSGKY7XLQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.451259425 (601h46m54.334673202s) [noise=true , address=45.91.95.2:4028 , piece_id=WPSVDMJBZOXWJA6ZYTBYUKMHWQEVNWTUXZL3U4K5MAXQVHJW5SBQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518816025 (601h46m54.402229644s) [noise=true , address=79.127.201.215:4349 , piece_id=YUZGMQNUXBHPGVJAIWNGHGIXKZKQHEWXWIZYGSBP5W4235FJA2PA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: <nil> Jul 18 10:57:54.518860075 (601h46m54.402273706s) [seg_pos={0 0}, upload=481170]: Segment reached optimal threshold of 14 pieces. Jul 18 10:57:54.519304291 (601h46m54.402717909s) [noise=true , address=79.127.205.234:4617 , piece_id=EYKK7F2UJYXWJPR6T2KXUPWW6KBZ22YIZETLYDZPF5GSLXAFNICA, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUJcG8EBim135i6mRQQrwSBS5CFqVN5ipdFsGdWYH5Y9CxaUe): protocol: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49788->79.127.205.234:4617: use of closed network connection Jul 18 10:57:54.519319363 (601h46m54.402732994s) [noise=true , address=79.127.205.235:4685 , piece_id=PUFURTT4HVKI3VSZP3X33VD7ULRNLCNMXRMNKX4VZXF5ED5E6CSQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UK3CA8sureHVG8LghUPqYxYQgTY4gY7gaejKaMSBjbpY4kc1M): protocol: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53700->79.127.205.235:4685: use of closed network connection Jul 18 10:57:54.519384628 (601h46m54.402798246s) [noise=true , address=79.127.205.233:4647 , piece_id=NPEUREZXRRSGDPKJ5USSSRIOXEOYBYCRQVBAYNODPIZ7ABC47LWQ, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c19Vd3TVx59sX7RWYHvY99eMUQRqjJCyryCMZ5NdSdqeDQu9s): protocol: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46308->79.127.205.233:4647: use of closed network connection Jul 18 10:57:54.519402736 (601h46m54.402816368s) [noise=true , address=77.90.31.15:4704 , piece_id=NPGY5BTN46QRMZZXZHML3GZYGVBBFFTBSSCY4VGDCGGBKOSGFQ3Q, seg_pos={0 0}, upload=481170]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj): protocol: write tcp 139.59.144.22:58638->77.90.31.15:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:57:54.519432414 (601h46m54.402846045s) [seg_pos={0 0}, upload=481170]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 18 10:57:54.519523054 (601h46m54.40293687s) [upload=481170]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:57:54.519529627 (601h46m54.402943243s) [upload=481170]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:57:54.662927129 (601h46m54.546340748s) [upload=481170]: Done uploading segments... Jul 18 10:57:54.662950246 (601h46m54.546363866s) [upload=481170]: Upload finished. err: <nil> Duration: 2.609100624s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....................X.....X.X..........X...............X...] duration: 2.34901559s FAIL (2025-07-18 11:27:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 139.59.144.22:52386->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:25:03.756616000 (602h14m3.640029628s) [seg_pos={0 0}, upload=481509]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:25:03.756750542 (602h14m3.640164173s) [upload=481509]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:25:03.756757773 (602h14m3.640171571s) [upload=481509]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:25:03.918610530 (602h14m3.802024164s) [upload=481509]: Done uploading segments... Jul 18 11:25:03.918632896 (602h14m3.802046512s) [upload=481509]: Upload finished. err: <nil> Jul 18 11:25:56.053888090 (602h14m55.937301716s) [upload=481521]: Starting upload Jul 18 11:25:56.053994103 (602h14m55.937407737s) [upload=481521]: Done starting upload Jul 18 11:25:56.054129122 (602h14m55.937542749s) [upload=481521]: Uploading segments... Jul 18 11:25:56.054306891 (602h14m55.937720517s) [upload=481521]: Got next segment. Inline: false Jul 18 11:25:56.054364770 (602h14m55.937778397s) [upload=481521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:25:56.054371001 (602h14m55.937784628s) [upload=481521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:25:56.170318697 (602h14m56.053732323s) [seg_pos={0 0}, upload=481521]: Begin upload segment... Jul 18 11:25:56.170422900 (602h14m56.053836526s) [seg_pos={0 0}, upload=481521]: Done begin upload segment. Jul 18 11:25:56.170464681 (602h14m56.053878314s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170521320 (602h14m56.053934956s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170853558 (602h14m56.054267185s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170909345 (602h14m56.054322977s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171219719 (602h14m56.054633352s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171337279 (602h14m56.054750915s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171541852 (602h14m56.054955482s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171704517 (602h14m56.055118151s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171920013 (602h14m56.055333643s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172277758 (602h14m56.055691391s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172326432 (602h14m56.055740064s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172568781 (602h14m56.055982394s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172668061 (602h14m56.056081694s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172832081 (602h14m56.056245713s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173001844 (602h14m56.056415473s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173339770 (602h14m56.056753404s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.786495143 (602h14m56.669908759s) [upload=481521]: Next returned nil segment Jul 18 11:25:56.786512550 (602h14m56.669926162s) [upload=481521]: Waiting for error group managing segments... Jul 18 11:25:56.918094694 (602h14m56.80150833s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.918628075 (602h14m56.802041694s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.919912323 (602h14m56.803325943s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.920806034 (602h14m56.804219647s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.921420465 (602h14m56.804834084s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.923349719 (602h14m56.80676334s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.926920740 (602h14m56.810334354s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.930546114 (602h14m56.813959734s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.931434176 (602h14m56.814847796s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.945660373 (602h14m56.829074004s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.955452695 (602h14m56.838866315s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.957436329 (602h14m56.840849962s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:26:25.766597801 (602h15m25.65001142s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766639011 (602h15m25.650052626s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:53448->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766658832 (602h15m25.650072444s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766671419 (602h15m25.650085044s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:48546->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766698381 (602h15m25.650112007s) [seg_pos={0 0}, upload=481521]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 11:26:25.884294106 (602h15m25.76770774s) [upload=481521]: Done uploading segments... Jul 18 11:26:25.884340118 (602h15m25.767753759s) [upload=481521]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.349045268s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [......................X.....X.X..........X...............X..] duration: 2.371254322s FAIL (2025-07-18 11:27:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 139.59.144.22:52386->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:25:03.756616000 (602h14m3.640029628s) [seg_pos={0 0}, upload=481509]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 11:25:03.756750542 (602h14m3.640164173s) [upload=481509]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 11:25:03.756757773 (602h14m3.640171571s) [upload=481509]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 11:25:03.918610530 (602h14m3.802024164s) [upload=481509]: Done uploading segments... Jul 18 11:25:03.918632896 (602h14m3.802046512s) [upload=481509]: Upload finished. err: <nil> Jul 18 11:25:56.053888090 (602h14m55.937301716s) [upload=481521]: Starting upload Jul 18 11:25:56.053994103 (602h14m55.937407737s) [upload=481521]: Done starting upload Jul 18 11:25:56.054129122 (602h14m55.937542749s) [upload=481521]: Uploading segments... Jul 18 11:25:56.054306891 (602h14m55.937720517s) [upload=481521]: Got next segment. Inline: false Jul 18 11:25:56.054364770 (602h14m55.937778397s) [upload=481521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 11:25:56.054371001 (602h14m55.937784628s) [upload=481521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 11:25:56.170318697 (602h14m56.053732323s) [seg_pos={0 0}, upload=481521]: Begin upload segment... Jul 18 11:25:56.170422900 (602h14m56.053836526s) [seg_pos={0 0}, upload=481521]: Done begin upload segment. Jul 18 11:25:56.170464681 (602h14m56.053878314s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170521320 (602h14m56.053934956s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170853558 (602h14m56.054267185s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.170909345 (602h14m56.054322977s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171219719 (602h14m56.054633352s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171337279 (602h14m56.054750915s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171541852 (602h14m56.054955482s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171704517 (602h14m56.055118151s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.171920013 (602h14m56.055333643s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172277758 (602h14m56.055691391s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172326432 (602h14m56.055740064s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172568781 (602h14m56.055982394s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172668061 (602h14m56.056081694s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.172832081 (602h14m56.056245713s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173001844 (602h14m56.056415473s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.173339770 (602h14m56.056753404s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Uploading piece... Jul 18 11:25:56.786495143 (602h14m56.669908759s) [upload=481521]: Next returned nil segment Jul 18 11:25:56.786512550 (602h14m56.669926162s) [upload=481521]: Waiting for error group managing segments... Jul 18 11:25:56.918094694 (602h14m56.80150833s) [noise=true , address=77.90.31.14:4663 , piece_id=2Z5MCGMNNK3T6EZAXQ4VCCSB43G7Q4XCFUQCLWISLD7SEGJ4ULRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.918628075 (602h14m56.802041694s) [noise=true , address=45.91.116.7:4010 , piece_id=MAWSNBI5PX5TUVXMZKWW3IIY2DJH52DC7WWDR4JH22G5W6FSRNYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.919912323 (602h14m56.803325943s) [noise=true , address=45.91.116.4:4937 , piece_id=63BWAY7FFF7JIQGAQDZFIRUTGFZGVRGO3DBUOGUTBOQVQDUDXYWA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.920806034 (602h14m56.804219647s) [noise=true , address=45.91.116.9:4262 , piece_id=B45IWUW3S4JK7RS6ZFOQATVQPA4XU2TCYEAR4BCCE7YOUFR6RVYQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.921420465 (602h14m56.804834084s) [noise=true , address=45.91.116.11:4117 , piece_id=3OCYS5VFJMWATMYL6XCWLHJZZEQVS4B3KL3BF25HUHKYRXAJXTTQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.923349719 (602h14m56.80676334s) [noise=true , address=77.90.31.12:4592 , piece_id=KZHGR2XLQ3E6WIOGR7FOJCGZ2BVFYCHFJST47MTU2WMP3GWBKFSQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.926920740 (602h14m56.810334354s) [noise=true , address=45.91.116.14:4190 , piece_id=BSG3QNFIZCD6IZB4YDHBX6IP364ERRCM44XAIBGS4OCAHNZSKKYA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.930546114 (602h14m56.813959734s) [noise=true , address=45.91.116.14:4191 , piece_id=Z4A45KI4M5WRB7W6SU2M4HEQMF5JET2XP4DHKV34IAOCLFBYODLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.931434176 (602h14m56.814847796s) [noise=true , address=45.91.116.11:4118 , piece_id=GWFR6OFDCHQ7OT6SDRKKNQIDIFN6HLNDLJANOFAUMB5GOPWRHV7A, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.945660373 (602h14m56.829074004s) [noise=true , address=104.36.82.5:4839 , piece_id=LXOMMJNPINRKDWRSG6A4KZCK6CZIK7SXOYHT6JPP2ZWLXBXVWTEA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.955452695 (602h14m56.838866315s) [noise=true , address=104.36.82.2:4731 , piece_id=3G357TEQIYMJ5ERZC6HQCJ34CSUJJK7TCPCPHQBXYTBFIRWECNLA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:25:56.957436329 (602h14m56.840849962s) [noise=true , address=104.36.82.4:4804 , piece_id=NVL5PYUVLYTGIQEXQ53WUTGOS4Q3FHBOIC62DKG2PG7ISE2LXKRA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: <nil> Jul 18 11:26:25.766597801 (602h15m25.65001142s) [noise=true , address=77.90.31.17:4772 , piece_id=BEAZBVRX4ZLH2YHRWLXV6QQFPEND5JSIR65UYQ5RAG2GIHSXX7BQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766639011 (602h15m25.650052626s) [noise=true , address=77.90.31.18:4807 , piece_id=E5UCGPVIBBK4L2BGUP4D53V3KUKWANC7YQWA3ZQVSUJMW6QEULGA, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 139.59.144.22:53448->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766658832 (602h15m25.650072444s) [noise=true , address=77.90.31.15:4698 , piece_id=GRKJTRYGNZ65V2TGS7EY5RZAHTKMWYUJRX3XHFG6F6IP42TS6BZQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766671419 (602h15m25.650085044s) [noise=true , address=77.90.31.16:4733 , piece_id=H33I77RZOSI5YTFJX2TRMPHRUGBZ6VC237BCWYGJQGFWYLJSG7KQ, seg_pos={0 0}, upload=481521]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:48546->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 11:26:25.766698381 (602h15m25.650112007s) [seg_pos={0 0}, upload=481521]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 11:26:25.884294106 (602h15m25.76770774s) [upload=481521]: Done uploading segments... Jul 18 11:26:25.884340118 (602h15m25.767753759s) [upload=481521]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.37129407s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)