Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................................................XXXX...] duration: 2.875192672s FAIL (2025-08-17 04:24:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277217340 (202h56m58.236314588s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277444018 (202h56m58.236541264s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277693944 (202h56m58.236791191s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277957287 (202h56m58.237054531s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278192843 (202h56m58.237290088s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278363507 (202h56m58.237460782s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278887870 (202h56m58.237985121s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279265735 (202h56m58.238362991s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279639846 (202h56m58.238737091s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279971414 (202h56m58.239068662s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280041108 (202h56m58.239138356s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280352792 (202h56m58.239450046s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280683314 (202h56m58.239780567s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280969649 (202h56m58.240066896s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281241193 (202h56m58.240338438s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281508639 (202h56m58.240605887s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.837376257 (202h56m58.796473489s) [upload=211240]: Next returned nil segment Aug 17 04:22:36.837399552 (202h56m58.796496798s) [upload=211240]: Waiting for error group managing segments... Aug 17 04:22:36.989047654 (202h56m58.948144887s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991378317 (202h56m58.950475552s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991998552 (202h56m58.951095785s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.010955158 (202h56m58.970052395s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.017235751 (202h56m58.97633299s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.045617264 (202h56m59.004714496s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.080535376 (202h56m59.039632621s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.121242355 (202h56m59.080339591s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.296208684 (202h56m59.255305934s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.528205336 (202h56m59.487302581s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:23:35.761418960 (202h57m57.720516213s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1okrDyM4TfzYN3p8zUjTqbw2661rKQQqZdP1ptAKzzTQ82C7cN, address:104.36.82.60:4341): protocol: rpc: dial tcp 104.36.82.60:4341: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761509741 (202h57m57.720606987s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1i62GRFASf7UPVGhUofmRkz3zZSyhq4fDycgyYtfD7pHR3w91v, address:104.36.82.34:4556): protocol: write tcp 139.59.144.22:37150->104.36.82.34:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761574924 (202h57m57.720672174s) [noise=true , address=104.36.82.17:4333 , piece_id=CGHZEP2ZKWES3L6XH4LSFLOFKYLQPYJZ56G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1xRdTZoEk3NsnS445GcdEvwAdDzNLwUYSfgjB7bBSGRx9L9mrU, address:104.36.82.17:4333): protocol: write tcp 139.59.144.22:47980->104.36.82.17:4333: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761620359 (202h57m57.720717605s) [noise=true , address=104.36.82.58:4293 , piece_id=IY3EUM3SZ42OZA42BC2FSVTLTHU5POSMOOQT6MTLHNPZ6FEYE2EQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:15i783VxxzbriD7LZCbFWNBY8uCKUffXtmWE2eu2VupgE1t9bZ, address:104.36.82.58:4293): protocol: write tcp 139.59.144.22:39172->104.36.82.58:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761661063 (202h57m57.720758308s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:12aETKgTzKw8fyYqbeDA2JzfxZnWn12xUPYK1eRt6TvBPPdRJDp, address:104.36.82.36:4630): protocol: write tcp 139.59.144.22:51970->104.36.82.36:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761681516 (202h57m57.720778766s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 139.59.144.22:35702->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761728458 (202h57m57.720825693s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:48846->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761769868 (202h57m57.720867115s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1hbt8HKTunpPKyAzH4XBwGQqTKzcgqPpoWwKZm9JMtaK8P7aVp, address:104.36.82.8:4992): protocol: write tcp 139.59.144.22:40976->104.36.82.8:4992: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761795616 (202h57m57.720892851s) [seg_pos={0 0}, upload=211240]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:35.867721911 (202h57m57.826819144s) [upload=211240]: Done uploading segments... Aug 17 04:23:35.867749737 (202h57m57.826846969s) [upload=211240]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 2.875319583s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [X....................................................XXX....] duration: 2.768477339s FAIL (2025-08-17 04:24:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 17 04:21:58.824378996 (202h56m20.783476241s) [upload=211227]: Starting upload Aug 17 04:21:58.824453728 (202h56m20.783550973s) [upload=211227]: Done starting upload Aug 17 04:21:58.824533712 (202h56m20.783630943s) [upload=211227]: Uploading segments... Aug 17 04:21:58.824814622 (202h56m20.783911863s) [upload=211227]: Got next segment. Inline: false Aug 17 04:21:58.824870724 (202h56m20.78396797s) [upload=211227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:21:58.824876461 (202h56m20.783973706s) [upload=211227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:21:59.025160521 (202h56m20.984257797s) [seg_pos={0 0}, upload=211227]: Begin upload segment... Aug 17 04:21:59.025238019 (202h56m20.984335266s) [seg_pos={0 0}, upload=211227]: Done begin upload segment. Aug 17 04:21:59.025273686 (202h56m20.98437092s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025331026 (202h56m20.984428309s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025756251 (202h56m20.984853496s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025907425 (202h56m20.98500468s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026127117 (202h56m20.98522437s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026350346 (202h56m20.985447594s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026436670 (202h56m20.985533916s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026714369 (202h56m20.985811614s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026788896 (202h56m20.985886144s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027068637 (202h56m20.986165882s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027120600 (202h56m20.986217846s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027420110 (202h56m20.986517355s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027425372 (202h56m20.98652262s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027839122 (202h56m20.986936376s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.028247142 (202h56m20.987344396s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030005732 (202h56m20.989102986s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030432900 (202h56m20.989530147s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.031846491 (202h56m20.990943746s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.588920323 (202h56m21.548017569s) [upload=211227]: Next returned nil segment Aug 17 04:21:59.588944741 (202h56m21.548041972s) [upload=211227]: Waiting for error group managing segments... Aug 17 04:21:59.741262552 (202h56m21.700359791s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.742768290 (202h56m21.701865528s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.777622259 (202h56m21.736719503s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.808692811 (202h56m21.767790057s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.831688332 (202h56m21.790785579s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.871745680 (202h56m21.830842918s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.883314367 (202h56m21.84241162s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.890311438 (202h56m21.849408688s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.940441979 (202h56m21.899539228s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.976656973 (202h56m21.935754218s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.283185203 (202h56m22.242282448s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.284822985 (202h56m22.243920238s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.446988508 (202h56m22.406085753s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:58.545488378 (202h57m20.504585625s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr, address:104.36.82.5:4866): protocol: write tcp 139.59.144.22:57798->104.36.82.5:4866: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545674721 (202h57m20.504771972s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 139.59.144.22:57190->104.36.82.23:4447: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545824338 (202h57m20.504921585s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1, address:104.36.82.44:4901): protocol: write tcp 139.59.144.22:54098->104.36.82.44:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545853626 (202h57m20.504950858s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12ASAeTMBu4sY1GuZiSuwTXmhPksA1Te2BVZdcoE9QvpLAmcQX3, address:104.36.82.35:4603): protocol: write tcp 139.59.144.22:49164->104.36.82.35:4603: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545907146 (202h57m20.505004393s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: write tcp 139.59.144.22:43990->104.36.82.60:4365: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545989950 (202h57m20.505087195s) [seg_pos={0 0}, upload=211227]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 04:22:58.646357515 (202h57m20.605454767s) [upload=211227]: Done uploading segments... Aug 17 04:22:58.646391150 (202h57m20.605488398s) [upload=211227]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.768510649s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................XX...] duration: 2.792736344s FAIL (2025-08-17 04:24:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.792782405s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [......................................................XXXXX.] duration: 2.756122015s FAIL (2025-08-17 04:24:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.756218427s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................................................XXXX..] duration: 3.61870947s FAIL (2025-08-17 04:24:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277217340 (202h56m58.236314588s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277444018 (202h56m58.236541264s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277693944 (202h56m58.236791191s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277957287 (202h56m58.237054531s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278192843 (202h56m58.237290088s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278363507 (202h56m58.237460782s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278887870 (202h56m58.237985121s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279265735 (202h56m58.238362991s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279639846 (202h56m58.238737091s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279971414 (202h56m58.239068662s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280041108 (202h56m58.239138356s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280352792 (202h56m58.239450046s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280683314 (202h56m58.239780567s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280969649 (202h56m58.240066896s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281241193 (202h56m58.240338438s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281508639 (202h56m58.240605887s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.837376257 (202h56m58.796473489s) [upload=211240]: Next returned nil segment Aug 17 04:22:36.837399552 (202h56m58.796496798s) [upload=211240]: Waiting for error group managing segments... Aug 17 04:22:36.989047654 (202h56m58.948144887s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991378317 (202h56m58.950475552s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991998552 (202h56m58.951095785s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.010955158 (202h56m58.970052395s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.017235751 (202h56m58.97633299s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.045617264 (202h56m59.004714496s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.080535376 (202h56m59.039632621s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.121242355 (202h56m59.080339591s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.296208684 (202h56m59.255305934s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.528205336 (202h56m59.487302581s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:23:35.761418960 (202h57m57.720516213s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1okrDyM4TfzYN3p8zUjTqbw2661rKQQqZdP1ptAKzzTQ82C7cN, address:104.36.82.60:4341): protocol: rpc: dial tcp 104.36.82.60:4341: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761509741 (202h57m57.720606987s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1i62GRFASf7UPVGhUofmRkz3zZSyhq4fDycgyYtfD7pHR3w91v, address:104.36.82.34:4556): protocol: write tcp 139.59.144.22:37150->104.36.82.34:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761574924 (202h57m57.720672174s) [noise=true , address=104.36.82.17:4333 , piece_id=CGHZEP2ZKWES3L6XH4LSFLOFKYLQPYJZ56G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1xRdTZoEk3NsnS445GcdEvwAdDzNLwUYSfgjB7bBSGRx9L9mrU, address:104.36.82.17:4333): protocol: write tcp 139.59.144.22:47980->104.36.82.17:4333: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761620359 (202h57m57.720717605s) [noise=true , address=104.36.82.58:4293 , piece_id=IY3EUM3SZ42OZA42BC2FSVTLTHU5POSMOOQT6MTLHNPZ6FEYE2EQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:15i783VxxzbriD7LZCbFWNBY8uCKUffXtmWE2eu2VupgE1t9bZ, address:104.36.82.58:4293): protocol: write tcp 139.59.144.22:39172->104.36.82.58:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761661063 (202h57m57.720758308s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:12aETKgTzKw8fyYqbeDA2JzfxZnWn12xUPYK1eRt6TvBPPdRJDp, address:104.36.82.36:4630): protocol: write tcp 139.59.144.22:51970->104.36.82.36:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761681516 (202h57m57.720778766s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 139.59.144.22:35702->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761728458 (202h57m57.720825693s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:48846->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761769868 (202h57m57.720867115s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1hbt8HKTunpPKyAzH4XBwGQqTKzcgqPpoWwKZm9JMtaK8P7aVp, address:104.36.82.8:4992): protocol: write tcp 139.59.144.22:40976->104.36.82.8:4992: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761795616 (202h57m57.720892851s) [seg_pos={0 0}, upload=211240]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:35.867721911 (202h57m57.826819144s) [upload=211240]: Done uploading segments... Aug 17 04:23:35.867749737 (202h57m57.826846969s) [upload=211240]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 3.618820878s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X....................................................XXX...] duration: 2.631746835s FAIL (2025-08-17 04:23:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 17 04:21:58.824378996 (202h56m20.783476241s) [upload=211227]: Starting upload Aug 17 04:21:58.824453728 (202h56m20.783550973s) [upload=211227]: Done starting upload Aug 17 04:21:58.824533712 (202h56m20.783630943s) [upload=211227]: Uploading segments... Aug 17 04:21:58.824814622 (202h56m20.783911863s) [upload=211227]: Got next segment. Inline: false Aug 17 04:21:58.824870724 (202h56m20.78396797s) [upload=211227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:21:58.824876461 (202h56m20.783973706s) [upload=211227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:21:59.025160521 (202h56m20.984257797s) [seg_pos={0 0}, upload=211227]: Begin upload segment... Aug 17 04:21:59.025238019 (202h56m20.984335266s) [seg_pos={0 0}, upload=211227]: Done begin upload segment. Aug 17 04:21:59.025273686 (202h56m20.98437092s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025331026 (202h56m20.984428309s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025756251 (202h56m20.984853496s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025907425 (202h56m20.98500468s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026127117 (202h56m20.98522437s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026350346 (202h56m20.985447594s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026436670 (202h56m20.985533916s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026714369 (202h56m20.985811614s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026788896 (202h56m20.985886144s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027068637 (202h56m20.986165882s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027120600 (202h56m20.986217846s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027420110 (202h56m20.986517355s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027425372 (202h56m20.98652262s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027839122 (202h56m20.986936376s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.028247142 (202h56m20.987344396s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030005732 (202h56m20.989102986s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030432900 (202h56m20.989530147s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.031846491 (202h56m20.990943746s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.588920323 (202h56m21.548017569s) [upload=211227]: Next returned nil segment Aug 17 04:21:59.588944741 (202h56m21.548041972s) [upload=211227]: Waiting for error group managing segments... Aug 17 04:21:59.741262552 (202h56m21.700359791s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.742768290 (202h56m21.701865528s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.777622259 (202h56m21.736719503s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.808692811 (202h56m21.767790057s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.831688332 (202h56m21.790785579s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.871745680 (202h56m21.830842918s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.883314367 (202h56m21.84241162s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.890311438 (202h56m21.849408688s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.940441979 (202h56m21.899539228s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.976656973 (202h56m21.935754218s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.283185203 (202h56m22.242282448s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.284822985 (202h56m22.243920238s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.446988508 (202h56m22.406085753s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:58.545488378 (202h57m20.504585625s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr, address:104.36.82.5:4866): protocol: write tcp 139.59.144.22:57798->104.36.82.5:4866: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545674721 (202h57m20.504771972s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 139.59.144.22:57190->104.36.82.23:4447: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545824338 (202h57m20.504921585s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1, address:104.36.82.44:4901): protocol: write tcp 139.59.144.22:54098->104.36.82.44:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545853626 (202h57m20.504950858s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12ASAeTMBu4sY1GuZiSuwTXmhPksA1Te2BVZdcoE9QvpLAmcQX3, address:104.36.82.35:4603): protocol: write tcp 139.59.144.22:49164->104.36.82.35:4603: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545907146 (202h57m20.505004393s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: write tcp 139.59.144.22:43990->104.36.82.60:4365: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545989950 (202h57m20.505087195s) [seg_pos={0 0}, upload=211227]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 04:22:58.646357515 (202h57m20.605454767s) [upload=211227]: Done uploading segments... Aug 17 04:22:58.646391150 (202h57m20.605488398s) [upload=211227]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.631780264s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................................XXXX.] duration: 2.473947835s FAIL (2025-08-17 04:23:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Aug 17 04:22:36.193175039 (202h56m58.152272297s) [upload=211241]: Starting upload Aug 17 04:22:36.193282733 (202h56m58.152379987s) [upload=211241]: Done starting upload Aug 17 04:22:36.193533603 (202h56m58.152630849s) [upload=211241]: Uploading segments... Aug 17 04:22:36.193824638 (202h56m58.152921893s) [upload=211241]: Got next segment. Inline: false Aug 17 04:22:36.193891646 (202h56m58.152988897s) [upload=211241]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:22:36.193898487 (202h56m58.15299574s) [upload=211241]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:22:36.387313731 (202h56m58.346410984s) [seg_pos={0 0}, upload=211241]: Begin upload segment... Aug 17 04:22:36.387391508 (202h56m58.346488754s) [seg_pos={0 0}, upload=211241]: Done begin upload segment. Aug 17 04:22:36.387453494 (202h56m58.34655085s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.387549035 (202h56m58.346646289s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388292176 (202h56m58.347389421s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388564570 (202h56m58.347661824s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388775483 (202h56m58.347872728s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388976887 (202h56m58.348074132s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389048807 (202h56m58.348146055s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389269495 (202h56m58.348366743s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389303339 (202h56m58.348400583s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389577693 (202h56m58.348674943s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389582267 (202h56m58.348679515s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389868029 (202h56m58.348965273s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389899606 (202h56m58.348996853s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390181444 (202h56m58.349278692s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390456412 (202h56m58.349553657s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.396850594 (202h56m58.355947842s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:37.003486038 (202h56m58.962583287s) [upload=211241]: Next returned nil segment Aug 17 04:22:37.003509888 (202h56m58.962607148s) [upload=211241]: Waiting for error group managing segments... Aug 17 04:22:37.141087302 (202h56m59.100184557s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.147724589 (202h56m59.106821827s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148191395 (202h56m59.107288641s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148249179 (202h56m59.107346437s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.154197920 (202h56m59.113295159s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155088755 (202h56m59.11418599s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155737855 (202h56m59.114835102s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.158542805 (202h56m59.117640044s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.163837059 (202h56m59.122934309s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.225669823 (202h56m59.184767059s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.249261694 (202h56m59.20835894s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:23:05.920185881 (202h57m27.879283118s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: rpc: dial tcp 104.36.82.10:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920277695 (202h57m27.879374941s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: rpc: dial tcp 104.36.82.15:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920476811 (202h57m27.87957406s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: rpc: dial tcp 104.36.82.7:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920550680 (202h57m27.879647915s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920687798 (202h57m27.879785037s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:52446->104.36.82.8:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 04:23:05.920736972 (202h57m27.879834217s) [seg_pos={0 0}, upload=211241]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Aug 17 04:23:06.024578477 (202h57m27.983675715s) [upload=211241]: Done uploading segments... Aug 17 04:23:06.024615107 (202h57m27.983712354s) [upload=211241]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.47397594s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................................XXXX.] duration: 2.625571251s FAIL (2025-08-17 04:23:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277217340 (202h56m58.236314588s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277444018 (202h56m58.236541264s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277693944 (202h56m58.236791191s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277957287 (202h56m58.237054531s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278192843 (202h56m58.237290088s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278363507 (202h56m58.237460782s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278887870 (202h56m58.237985121s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279265735 (202h56m58.238362991s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279639846 (202h56m58.238737091s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279971414 (202h56m58.239068662s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280041108 (202h56m58.239138356s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280352792 (202h56m58.239450046s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280683314 (202h56m58.239780567s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280969649 (202h56m58.240066896s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281241193 (202h56m58.240338438s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281508639 (202h56m58.240605887s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.837376257 (202h56m58.796473489s) [upload=211240]: Next returned nil segment Aug 17 04:22:36.837399552 (202h56m58.796496798s) [upload=211240]: Waiting for error group managing segments... Aug 17 04:22:36.989047654 (202h56m58.948144887s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991378317 (202h56m58.950475552s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991998552 (202h56m58.951095785s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.010955158 (202h56m58.970052395s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.017235751 (202h56m58.97633299s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.045617264 (202h56m59.004714496s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.080535376 (202h56m59.039632621s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.121242355 (202h56m59.080339591s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.296208684 (202h56m59.255305934s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.528205336 (202h56m59.487302581s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:23:35.761418960 (202h57m57.720516213s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1okrDyM4TfzYN3p8zUjTqbw2661rKQQqZdP1ptAKzzTQ82C7cN, address:104.36.82.60:4341): protocol: rpc: dial tcp 104.36.82.60:4341: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761509741 (202h57m57.720606987s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1i62GRFASf7UPVGhUofmRkz3zZSyhq4fDycgyYtfD7pHR3w91v, address:104.36.82.34:4556): protocol: write tcp 139.59.144.22:37150->104.36.82.34:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761574924 (202h57m57.720672174s) [noise=true , address=104.36.82.17:4333 , piece_id=CGHZEP2ZKWES3L6XH4LSFLOFKYLQPYJZ56G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1xRdTZoEk3NsnS445GcdEvwAdDzNLwUYSfgjB7bBSGRx9L9mrU, address:104.36.82.17:4333): protocol: write tcp 139.59.144.22:47980->104.36.82.17:4333: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761620359 (202h57m57.720717605s) [noise=true , address=104.36.82.58:4293 , piece_id=IY3EUM3SZ42OZA42BC2FSVTLTHU5POSMOOQT6MTLHNPZ6FEYE2EQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:15i783VxxzbriD7LZCbFWNBY8uCKUffXtmWE2eu2VupgE1t9bZ, address:104.36.82.58:4293): protocol: write tcp 139.59.144.22:39172->104.36.82.58:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761661063 (202h57m57.720758308s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:12aETKgTzKw8fyYqbeDA2JzfxZnWn12xUPYK1eRt6TvBPPdRJDp, address:104.36.82.36:4630): protocol: write tcp 139.59.144.22:51970->104.36.82.36:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761681516 (202h57m57.720778766s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 139.59.144.22:35702->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761728458 (202h57m57.720825693s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:48846->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761769868 (202h57m57.720867115s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1hbt8HKTunpPKyAzH4XBwGQqTKzcgqPpoWwKZm9JMtaK8P7aVp, address:104.36.82.8:4992): protocol: write tcp 139.59.144.22:40976->104.36.82.8:4992: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761795616 (202h57m57.720892851s) [seg_pos={0 0}, upload=211240]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:35.867721911 (202h57m57.826819144s) [upload=211240]: Done uploading segments... Aug 17 04:23:35.867749737 (202h57m57.826846969s) [upload=211240]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 2.625613535s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..X....................................................XXX..] duration: 2.593503689s FAIL (2025-08-17 04:23:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 17 04:21:58.824378996 (202h56m20.783476241s) [upload=211227]: Starting upload Aug 17 04:21:58.824453728 (202h56m20.783550973s) [upload=211227]: Done starting upload Aug 17 04:21:58.824533712 (202h56m20.783630943s) [upload=211227]: Uploading segments... Aug 17 04:21:58.824814622 (202h56m20.783911863s) [upload=211227]: Got next segment. Inline: false Aug 17 04:21:58.824870724 (202h56m20.78396797s) [upload=211227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:21:58.824876461 (202h56m20.783973706s) [upload=211227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:21:59.025160521 (202h56m20.984257797s) [seg_pos={0 0}, upload=211227]: Begin upload segment... Aug 17 04:21:59.025238019 (202h56m20.984335266s) [seg_pos={0 0}, upload=211227]: Done begin upload segment. Aug 17 04:21:59.025273686 (202h56m20.98437092s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025331026 (202h56m20.984428309s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025756251 (202h56m20.984853496s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025907425 (202h56m20.98500468s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026127117 (202h56m20.98522437s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026350346 (202h56m20.985447594s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026436670 (202h56m20.985533916s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026714369 (202h56m20.985811614s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026788896 (202h56m20.985886144s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027068637 (202h56m20.986165882s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027120600 (202h56m20.986217846s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027420110 (202h56m20.986517355s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027425372 (202h56m20.98652262s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027839122 (202h56m20.986936376s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.028247142 (202h56m20.987344396s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030005732 (202h56m20.989102986s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030432900 (202h56m20.989530147s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.031846491 (202h56m20.990943746s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.588920323 (202h56m21.548017569s) [upload=211227]: Next returned nil segment Aug 17 04:21:59.588944741 (202h56m21.548041972s) [upload=211227]: Waiting for error group managing segments... Aug 17 04:21:59.741262552 (202h56m21.700359791s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.742768290 (202h56m21.701865528s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.777622259 (202h56m21.736719503s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.808692811 (202h56m21.767790057s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.831688332 (202h56m21.790785579s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.871745680 (202h56m21.830842918s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.883314367 (202h56m21.84241162s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.890311438 (202h56m21.849408688s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.940441979 (202h56m21.899539228s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.976656973 (202h56m21.935754218s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.283185203 (202h56m22.242282448s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.284822985 (202h56m22.243920238s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.446988508 (202h56m22.406085753s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:58.545488378 (202h57m20.504585625s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr, address:104.36.82.5:4866): protocol: write tcp 139.59.144.22:57798->104.36.82.5:4866: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545674721 (202h57m20.504771972s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 139.59.144.22:57190->104.36.82.23:4447: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545824338 (202h57m20.504921585s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1, address:104.36.82.44:4901): protocol: write tcp 139.59.144.22:54098->104.36.82.44:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545853626 (202h57m20.504950858s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12ASAeTMBu4sY1GuZiSuwTXmhPksA1Te2BVZdcoE9QvpLAmcQX3, address:104.36.82.35:4603): protocol: write tcp 139.59.144.22:49164->104.36.82.35:4603: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545907146 (202h57m20.505004393s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: write tcp 139.59.144.22:43990->104.36.82.60:4365: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545989950 (202h57m20.505087195s) [seg_pos={0 0}, upload=211227]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 04:22:58.646357515 (202h57m20.605454767s) [upload=211227]: Done uploading segments... Aug 17 04:22:58.646391150 (202h57m20.605488398s) [upload=211227]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.593553858s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 1m0.108809751s FAIL (2025-08-17 04:23:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277217340 (202h56m58.236314588s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277444018 (202h56m58.236541264s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277693944 (202h56m58.236791191s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277957287 (202h56m58.237054531s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278192843 (202h56m58.237290088s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278363507 (202h56m58.237460782s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278887870 (202h56m58.237985121s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279265735 (202h56m58.238362991s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279639846 (202h56m58.238737091s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279971414 (202h56m58.239068662s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280041108 (202h56m58.239138356s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280352792 (202h56m58.239450046s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280683314 (202h56m58.239780567s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280969649 (202h56m58.240066896s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281241193 (202h56m58.240338438s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281508639 (202h56m58.240605887s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.837376257 (202h56m58.796473489s) [upload=211240]: Next returned nil segment Aug 17 04:22:36.837399552 (202h56m58.796496798s) [upload=211240]: Waiting for error group managing segments... Aug 17 04:22:36.989047654 (202h56m58.948144887s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991378317 (202h56m58.950475552s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991998552 (202h56m58.951095785s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.010955158 (202h56m58.970052395s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.017235751 (202h56m58.97633299s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.045617264 (202h56m59.004714496s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.080535376 (202h56m59.039632621s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.121242355 (202h56m59.080339591s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.296208684 (202h56m59.255305934s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.528205336 (202h56m59.487302581s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:23:35.761418960 (202h57m57.720516213s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1okrDyM4TfzYN3p8zUjTqbw2661rKQQqZdP1ptAKzzTQ82C7cN, address:104.36.82.60:4341): protocol: rpc: dial tcp 104.36.82.60:4341: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761509741 (202h57m57.720606987s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1i62GRFASf7UPVGhUofmRkz3zZSyhq4fDycgyYtfD7pHR3w91v, address:104.36.82.34:4556): protocol: write tcp 139.59.144.22:37150->104.36.82.34:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761574924 (202h57m57.720672174s) [noise=true , address=104.36.82.17:4333 , piece_id=CGHZEP2ZKWES3L6XH4LSFLOFKYLQPYJZ56G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1xRdTZoEk3NsnS445GcdEvwAdDzNLwUYSfgjB7bBSGRx9L9mrU, address:104.36.82.17:4333): protocol: write tcp 139.59.144.22:47980->104.36.82.17:4333: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761620359 (202h57m57.720717605s) [noise=true , address=104.36.82.58:4293 , piece_id=IY3EUM3SZ42OZA42BC2FSVTLTHU5POSMOOQT6MTLHNPZ6FEYE2EQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:15i783VxxzbriD7LZCbFWNBY8uCKUffXtmWE2eu2VupgE1t9bZ, address:104.36.82.58:4293): protocol: write tcp 139.59.144.22:39172->104.36.82.58:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761661063 (202h57m57.720758308s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:12aETKgTzKw8fyYqbeDA2JzfxZnWn12xUPYK1eRt6TvBPPdRJDp, address:104.36.82.36:4630): protocol: write tcp 139.59.144.22:51970->104.36.82.36:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761681516 (202h57m57.720778766s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 139.59.144.22:35702->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761728458 (202h57m57.720825693s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:48846->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761769868 (202h57m57.720867115s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1hbt8HKTunpPKyAzH4XBwGQqTKzcgqPpoWwKZm9JMtaK8P7aVp, address:104.36.82.8:4992): protocol: write tcp 139.59.144.22:40976->104.36.82.8:4992: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761795616 (202h57m57.720892851s) [seg_pos={0 0}, upload=211240]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:35.867721911 (202h57m57.826819144s) [upload=211240]: Done uploading segments... Aug 17 04:23:35.867749737 (202h57m57.826846969s) [upload=211240]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.108838498s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...X....................................................XXX.] duration: 34.955978592s FAIL (2025-08-17 04:23:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 17 04:21:58.824378996 (202h56m20.783476241s) [upload=211227]: Starting upload Aug 17 04:21:58.824453728 (202h56m20.783550973s) [upload=211227]: Done starting upload Aug 17 04:21:58.824533712 (202h56m20.783630943s) [upload=211227]: Uploading segments... Aug 17 04:21:58.824814622 (202h56m20.783911863s) [upload=211227]: Got next segment. Inline: false Aug 17 04:21:58.824870724 (202h56m20.78396797s) [upload=211227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:21:58.824876461 (202h56m20.783973706s) [upload=211227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:21:59.025160521 (202h56m20.984257797s) [seg_pos={0 0}, upload=211227]: Begin upload segment... Aug 17 04:21:59.025238019 (202h56m20.984335266s) [seg_pos={0 0}, upload=211227]: Done begin upload segment. Aug 17 04:21:59.025273686 (202h56m20.98437092s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025331026 (202h56m20.984428309s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025756251 (202h56m20.984853496s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025907425 (202h56m20.98500468s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026127117 (202h56m20.98522437s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026350346 (202h56m20.985447594s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026436670 (202h56m20.985533916s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026714369 (202h56m20.985811614s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026788896 (202h56m20.985886144s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027068637 (202h56m20.986165882s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027120600 (202h56m20.986217846s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027420110 (202h56m20.986517355s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027425372 (202h56m20.98652262s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027839122 (202h56m20.986936376s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.028247142 (202h56m20.987344396s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030005732 (202h56m20.989102986s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030432900 (202h56m20.989530147s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.031846491 (202h56m20.990943746s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.588920323 (202h56m21.548017569s) [upload=211227]: Next returned nil segment Aug 17 04:21:59.588944741 (202h56m21.548041972s) [upload=211227]: Waiting for error group managing segments... Aug 17 04:21:59.741262552 (202h56m21.700359791s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.742768290 (202h56m21.701865528s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.777622259 (202h56m21.736719503s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.808692811 (202h56m21.767790057s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.831688332 (202h56m21.790785579s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.871745680 (202h56m21.830842918s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.883314367 (202h56m21.84241162s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.890311438 (202h56m21.849408688s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.940441979 (202h56m21.899539228s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.976656973 (202h56m21.935754218s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.283185203 (202h56m22.242282448s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.284822985 (202h56m22.243920238s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.446988508 (202h56m22.406085753s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:58.545488378 (202h57m20.504585625s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr, address:104.36.82.5:4866): protocol: write tcp 139.59.144.22:57798->104.36.82.5:4866: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545674721 (202h57m20.504771972s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 139.59.144.22:57190->104.36.82.23:4447: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545824338 (202h57m20.504921585s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1, address:104.36.82.44:4901): protocol: write tcp 139.59.144.22:54098->104.36.82.44:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545853626 (202h57m20.504950858s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12ASAeTMBu4sY1GuZiSuwTXmhPksA1Te2BVZdcoE9QvpLAmcQX3, address:104.36.82.35:4603): protocol: write tcp 139.59.144.22:49164->104.36.82.35:4603: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545907146 (202h57m20.505004393s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: write tcp 139.59.144.22:43990->104.36.82.60:4365: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545989950 (202h57m20.505087195s) [seg_pos={0 0}, upload=211227]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 04:22:58.646357515 (202h57m20.605454767s) [upload=211227]: Done uploading segments... Aug 17 04:22:58.646391150 (202h57m20.605488398s) [upload=211227]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 34.95600553s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration: 30.001346304s FAIL (2025-08-17 04:23:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001372546s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................................................XX..] duration: 3.152170261s FAIL (2025-08-17 04:23:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.152244394s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 30.106818789s FAIL (2025-08-17 04:23:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Aug 17 04:22:36.193175039 (202h56m58.152272297s) [upload=211241]: Starting upload Aug 17 04:22:36.193282733 (202h56m58.152379987s) [upload=211241]: Done starting upload Aug 17 04:22:36.193533603 (202h56m58.152630849s) [upload=211241]: Uploading segments... Aug 17 04:22:36.193824638 (202h56m58.152921893s) [upload=211241]: Got next segment. Inline: false Aug 17 04:22:36.193891646 (202h56m58.152988897s) [upload=211241]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:22:36.193898487 (202h56m58.15299574s) [upload=211241]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:22:36.387313731 (202h56m58.346410984s) [seg_pos={0 0}, upload=211241]: Begin upload segment... Aug 17 04:22:36.387391508 (202h56m58.346488754s) [seg_pos={0 0}, upload=211241]: Done begin upload segment. Aug 17 04:22:36.387453494 (202h56m58.34655085s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.387549035 (202h56m58.346646289s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388292176 (202h56m58.347389421s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388564570 (202h56m58.347661824s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388775483 (202h56m58.347872728s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388976887 (202h56m58.348074132s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389048807 (202h56m58.348146055s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389269495 (202h56m58.348366743s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389303339 (202h56m58.348400583s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389577693 (202h56m58.348674943s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389582267 (202h56m58.348679515s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389868029 (202h56m58.348965273s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389899606 (202h56m58.348996853s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390181444 (202h56m58.349278692s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390456412 (202h56m58.349553657s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.396850594 (202h56m58.355947842s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:37.003486038 (202h56m58.962583287s) [upload=211241]: Next returned nil segment Aug 17 04:22:37.003509888 (202h56m58.962607148s) [upload=211241]: Waiting for error group managing segments... Aug 17 04:22:37.141087302 (202h56m59.100184557s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.147724589 (202h56m59.106821827s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148191395 (202h56m59.107288641s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148249179 (202h56m59.107346437s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.154197920 (202h56m59.113295159s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155088755 (202h56m59.11418599s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155737855 (202h56m59.114835102s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.158542805 (202h56m59.117640044s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.163837059 (202h56m59.122934309s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.225669823 (202h56m59.184767059s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.249261694 (202h56m59.20835894s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:23:05.920185881 (202h57m27.879283118s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: rpc: dial tcp 104.36.82.10:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920277695 (202h57m27.879374941s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: rpc: dial tcp 104.36.82.15:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920476811 (202h57m27.87957406s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: rpc: dial tcp 104.36.82.7:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920550680 (202h57m27.879647915s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920687798 (202h57m27.879785037s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:52446->104.36.82.8:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 04:23:05.920736972 (202h57m27.879834217s) [seg_pos={0 0}, upload=211241]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Aug 17 04:23:06.024578477 (202h57m27.983675715s) [upload=211241]: Done uploading segments... Aug 17 04:23:06.024615107 (202h57m27.983712354s) [upload=211241]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.106838617s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [....X....................................................XXX] duration: 1m0.102571273s FAIL (2025-08-17 04:22:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 17 04:21:58.824378996 (202h56m20.783476241s) [upload=211227]: Starting upload Aug 17 04:21:58.824453728 (202h56m20.783550973s) [upload=211227]: Done starting upload Aug 17 04:21:58.824533712 (202h56m20.783630943s) [upload=211227]: Uploading segments... Aug 17 04:21:58.824814622 (202h56m20.783911863s) [upload=211227]: Got next segment. Inline: false Aug 17 04:21:58.824870724 (202h56m20.78396797s) [upload=211227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:21:58.824876461 (202h56m20.783973706s) [upload=211227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:21:59.025160521 (202h56m20.984257797s) [seg_pos={0 0}, upload=211227]: Begin upload segment... Aug 17 04:21:59.025238019 (202h56m20.984335266s) [seg_pos={0 0}, upload=211227]: Done begin upload segment. Aug 17 04:21:59.025273686 (202h56m20.98437092s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025331026 (202h56m20.984428309s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025756251 (202h56m20.984853496s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025907425 (202h56m20.98500468s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026127117 (202h56m20.98522437s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026350346 (202h56m20.985447594s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026436670 (202h56m20.985533916s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026714369 (202h56m20.985811614s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026788896 (202h56m20.985886144s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027068637 (202h56m20.986165882s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027120600 (202h56m20.986217846s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027420110 (202h56m20.986517355s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027425372 (202h56m20.98652262s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027839122 (202h56m20.986936376s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.028247142 (202h56m20.987344396s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030005732 (202h56m20.989102986s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030432900 (202h56m20.989530147s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.031846491 (202h56m20.990943746s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.588920323 (202h56m21.548017569s) [upload=211227]: Next returned nil segment Aug 17 04:21:59.588944741 (202h56m21.548041972s) [upload=211227]: Waiting for error group managing segments... Aug 17 04:21:59.741262552 (202h56m21.700359791s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.742768290 (202h56m21.701865528s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.777622259 (202h56m21.736719503s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.808692811 (202h56m21.767790057s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.831688332 (202h56m21.790785579s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.871745680 (202h56m21.830842918s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.883314367 (202h56m21.84241162s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.890311438 (202h56m21.849408688s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.940441979 (202h56m21.899539228s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.976656973 (202h56m21.935754218s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.283185203 (202h56m22.242282448s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.284822985 (202h56m22.243920238s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.446988508 (202h56m22.406085753s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:58.545488378 (202h57m20.504585625s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr, address:104.36.82.5:4866): protocol: write tcp 139.59.144.22:57798->104.36.82.5:4866: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545674721 (202h57m20.504771972s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 139.59.144.22:57190->104.36.82.23:4447: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545824338 (202h57m20.504921585s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1, address:104.36.82.44:4901): protocol: write tcp 139.59.144.22:54098->104.36.82.44:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545853626 (202h57m20.504950858s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12ASAeTMBu4sY1GuZiSuwTXmhPksA1Te2BVZdcoE9QvpLAmcQX3, address:104.36.82.35:4603): protocol: write tcp 139.59.144.22:49164->104.36.82.35:4603: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545907146 (202h57m20.505004393s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: write tcp 139.59.144.22:43990->104.36.82.60:4365: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545989950 (202h57m20.505087195s) [seg_pos={0 0}, upload=211227]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 04:22:58.646357515 (202h57m20.605454767s) [upload=211227]: Done uploading segments... Aug 17 04:22:58.646391150 (202h57m20.605488398s) [upload=211227]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.102590594s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 30.000756195s FAIL (2025-08-17 04:22:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000822624s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 2.935714345s FAIL (2025-08-17 04:22:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.935735123s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 30.105698197s FAIL (2025-08-17 04:22:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 04:22:05.950813942 (202h56m27.909911189s) [upload=211230]: Starting upload Aug 17 04:22:05.950909611 (202h56m27.910006868s) [upload=211230]: Done starting upload Aug 17 04:22:05.951144012 (202h56m27.910241257s) [upload=211230]: Uploading segments... Aug 17 04:22:05.951414643 (202h56m27.910511892s) [upload=211230]: Got next segment. Inline: false Aug 17 04:22:05.951513907 (202h56m27.910611151s) [upload=211230]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:22:05.951536725 (202h56m27.910633974s) [upload=211230]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:22:06.169440136 (202h56m28.1285374s) [seg_pos={0 0}, upload=211230]: Begin upload segment... Aug 17 04:22:06.169501546 (202h56m28.12859878s) [seg_pos={0 0}, upload=211230]: Done begin upload segment. Aug 17 04:22:06.169539363 (202h56m28.128636617s) [noise=true , address=45.91.116.11:4119 , piece_id=MMRKMBSXW7SYHPR7TMTRI4SUMADYJ3CE6PD2ZFOUOQINIFNFTBLA, seg_pos={0 0}, upload=211230]: Uploading piece... Aug 17 04:22:06.169586531 (202h56m28.128683786s) [noise=true , address=104.36.82.13:4185 , piece_id=DEUW5K33UOJ5FYLNHEZXQR3WXAIGFKEU7AR2VLGVVPA6E52PSFPQ, seg_pos={0 0}, upload=211230]: Uploading piece... Aug 17 04:22:06.170055882 (202h56m28.12915313s) [noise=true , address=104.36.82.14:4222 , piece_id=VEYHUNQVBE22U3QLY7D5X72FFF2DNO7KVKXH5XK4UCZA43I2TE3A, seg_pos={0 0}, upload=211230]: Uploading piece... Aug 17 04:22:06.170130861 (202h56m28.12922812s) [noise=true , address=77.90.31.14:4661 , piece_id=DJYVJ6VCR3YH2LZP5RXTAEPYSD7DYCDOCQFSGCSYUPNDRELG72LA, seg_pos={0 0}, upload=211230]: Uploading piece... Aug 17 04:22:06.170572415 (202h56m28.12966967s) [noise=true , address=45.91.116.14:4190 , piece_id=Y4V2AMAMNR4NODTBWVGU6MCKQYN6HRXUFW6EYDK3FZYPINIU752Q, seg_pos={0 0}, upload=211230]: Uploading piece... Aug 17 04:22:06.171149522 (202h56m28.130246775s) [noise=true , address=104.36.82.2:4729 , piece_id=HSP5PHQTM2ZWHMRRS44TJ55YSIHRUPG6RKVZE4SCTSIFKHQI5OBA, seg_pos={0 0}, upload=211230]: Uploading piece... Aug 17 04:22:06.171274284 (202h56m28.130371529s) [noise=true , address=104.36.82.12:4151 , piece_id=O4DNYW5CGGEJDPHKKA3JJP22SZFKKVQWZYZEKDBP2AWHN2ROVHAQ, seg_pos={0 0}, upload=211230]: Uploading piece... Aug 17 04:22:06.171577123 (202h56m28.130674431s) [noise=true , address=104.36.82.6:4875 , piece_id=N46VTYFTPK2IQI3RCUQKXVFTP7IMUZNOHJGIMLQAYGGLBCIQ4WUQ, seg_pos={0 0}, upload=211230]: Uploading piece... Aug 17 04:22:06.171866281 (202h56m28.130963526s) [noise=true , address=45.91.116.6:4973 , piece_id=U2QVUJZSA36632JNPXOCJRRSEDDGMQY2R74JJEMC5PFW24FTCSBQ, seg_pos={0 0}, upload=211230]: Uploading piece... Aug 17 04:22:06.172140208 (202h56m28.131237459s) [noise=true , address=77.90.31.10:4517 , piece_id=PLRQBCTDUEJVHWSJRTXIIQBU6BOH2UWYUQNWNWR6YQJEJAMV3OFA, seg_pos={0 0}, upload=211230]: Uploading piece... Aug 17 04:22:06.172517535 (202h56m28.13161478s) [noise=true , address=77.90.31.12:4589 , piece_id=KV6V236KY5BTSTXXVQ7TE5P4JOTHJHJJOQVLFCWSZEMSZXNAMOXQ, seg_pos={0 0}, upload=211230]: Uploading piece... Aug 17 04:22:06.172766817 (202h56m28.131864065s) [noise=true , address=45.91.116.14:4189 , piece_id=T354PZDZUQVZ6T7XAHVUTATPT32X4HEPS4MWGTDTETZ4R63K2Q6A, seg_pos={0 0}, upload=211230]: Uploading piece... Aug 17 04:22:06.172801848 (202h56m28.1318991s) [noise=true , address=77.90.31.15:4699 , piece_id=VNTYQT6OVFLZEF2GAULGFVRWLUJ67OMX5FK4XIQHSHVEFNLXQCRA, seg_pos={0 0}, upload=211230]: Uploading piece... Aug 17 04:22:06.173207973 (202h56m28.132305229s) [noise=true , address=104.36.82.11:4115 , piece_id=KYG6MHA7FTJYC4NH2KIRMFXSKAAEO3ZP57S54RILM4AWSZ2BX4BQ, seg_pos={0 0}, upload=211230]: Uploading piece... Aug 17 04:22:06.173595967 (202h56m28.132693221s) [noise=true , address=77.90.31.16:4733 , piece_id=4JXDT53KJWEQCFZJTDJNKEFZIC32I4AZ2CZHFN6RIJYZ7H25FFEQ, seg_pos={0 0}, upload=211230]: Uploading piece... Aug 17 04:22:06.173864699 (202h56m28.132961954s) [noise=true , address=104.36.82.13:4187 , piece_id=MA4FYL6BHE63DWYSDVSENGXJRE2NPIAOB3IBS24SFPGZEKOTI4XQ, seg_pos={0 0}, upload=211230]: Uploading piece... Aug 17 04:22:06.778703401 (202h56m28.737800642s) [upload=211230]: Next returned nil segment Aug 17 04:22:06.778728339 (202h56m28.737825594s) [upload=211230]: Waiting for error group managing segments... Aug 17 04:22:06.914958299 (202h56m28.874055539s) [noise=true , address=45.91.116.14:4190 , piece_id=Y4V2AMAMNR4NODTBWVGU6MCKQYN6HRXUFW6EYDK3FZYPINIU752Q, seg_pos={0 0}, upload=211230]: Done uploading piece. err: <nil> Aug 17 04:22:06.939096948 (202h56m28.898194193s) [noise=true , address=77.90.31.16:4733 , piece_id=4JXDT53KJWEQCFZJTDJNKEFZIC32I4AZ2CZHFN6RIJYZ7H25FFEQ, seg_pos={0 0}, upload=211230]: Done uploading piece. err: <nil> Aug 17 04:22:06.939202899 (202h56m28.898300145s) [noise=true , address=45.91.116.6:4973 , piece_id=U2QVUJZSA36632JNPXOCJRRSEDDGMQY2R74JJEMC5PFW24FTCSBQ, seg_pos={0 0}, upload=211230]: Done uploading piece. err: <nil> Aug 17 04:22:06.943358399 (202h56m28.902455645s) [noise=true , address=45.91.116.11:4119 , piece_id=MMRKMBSXW7SYHPR7TMTRI4SUMADYJ3CE6PD2ZFOUOQINIFNFTBLA, seg_pos={0 0}, upload=211230]: Done uploading piece. err: <nil> Aug 17 04:22:06.943395686 (202h56m28.902492924s) [noise=true , address=77.90.31.14:4661 , piece_id=DJYVJ6VCR3YH2LZP5RXTAEPYSD7DYCDOCQFSGCSYUPNDRELG72LA, seg_pos={0 0}, upload=211230]: Done uploading piece. err: <nil> Aug 17 04:22:06.955664548 (202h56m28.914761787s) [noise=true , address=77.90.31.15:4699 , piece_id=VNTYQT6OVFLZEF2GAULGFVRWLUJ67OMX5FK4XIQHSHVEFNLXQCRA, seg_pos={0 0}, upload=211230]: Done uploading piece. err: <nil> Aug 17 04:22:06.969131483 (202h56m28.928228721s) [noise=true , address=77.90.31.12:4589 , piece_id=KV6V236KY5BTSTXXVQ7TE5P4JOTHJHJJOQVLFCWSZEMSZXNAMOXQ, seg_pos={0 0}, upload=211230]: Done uploading piece. err: <nil> Aug 17 04:22:06.999521789 (202h56m28.958619033s) [noise=true , address=45.91.116.14:4189 , piece_id=T354PZDZUQVZ6T7XAHVUTATPT32X4HEPS4MWGTDTETZ4R63K2Q6A, seg_pos={0 0}, upload=211230]: Done uploading piece. err: <nil> Aug 17 04:22:07.082888466 (202h56m29.041985715s) [noise=true , address=77.90.31.10:4517 , piece_id=PLRQBCTDUEJVHWSJRTXIIQBU6BOH2UWYUQNWNWR6YQJEJAMV3OFA, seg_pos={0 0}, upload=211230]: Done uploading piece. err: <nil> Aug 17 04:22:35.681456835 (202h56m57.640554073s) [noise=true , address=104.36.82.6:4875 , piece_id=N46VTYFTPK2IQI3RCUQKXVFTP7IMUZNOHJGIMLQAYGGLBCIQ4WUQ, seg_pos={0 0}, upload=211230]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:104.36.82.6:4875): protocol: rpc: dial tcp 104.36.82.6:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:35.681573079 (202h56m57.640670324s) [noise=true , address=104.36.82.13:4185 , piece_id=DEUW5K33UOJ5FYLNHEZXQR3WXAIGFKEU7AR2VLGVVPA6E52PSFPQ, seg_pos={0 0}, upload=211230]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: rpc: dial tcp 104.36.82.13:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:35.681630507 (202h56m57.640727755s) [noise=true , address=104.36.82.11:4115 , piece_id=KYG6MHA7FTJYC4NH2KIRMFXSKAAEO3ZP57S54RILM4AWSZ2BX4BQ, seg_pos={0 0}, upload=211230]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: rpc: dial tcp 104.36.82.11:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:35.681807688 (202h56m57.640904931s) [noise=true , address=104.36.82.14:4222 , piece_id=VEYHUNQVBE22U3QLY7D5X72FFF2DNO7KVKXH5XK4UCZA43I2TE3A, seg_pos={0 0}, upload=211230]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:35.681868399 (202h56m57.640965636s) [noise=true , address=104.36.82.13:4187 , piece_id=MA4FYL6BHE63DWYSDVSENGXJRE2NPIAOB3IBS24SFPGZEKOTI4XQ, seg_pos={0 0}, upload=211230]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:35.681888760 (202h56m57.640986006s) [noise=true , address=104.36.82.2:4729 , piece_id=HSP5PHQTM2ZWHMRRS44TJ55YSIHRUPG6RKVZE4SCTSIFKHQI5OBA, seg_pos={0 0}, upload=211230]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: rpc: dial tcp 104.36.82.2:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:35.681951701 (202h56m57.641048945s) [noise=true , address=104.36.82.12:4151 , piece_id=O4DNYW5CGGEJDPHKKA3JJP22SZFKKVQWZYZEKDBP2AWHN2ROVHAQ, seg_pos={0 0}, upload=211230]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: rpc: dial tcp 104.36.82.12:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:35.681979275 (202h56m57.641076508s) [seg_pos={0 0}, upload=211230]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 04:22:35.785218049 (202h56m57.744315299s) [upload=211230]: Done uploading segments... Aug 17 04:22:35.785244022 (202h56m57.744341267s) [upload=211230]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.105726494s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 1m0.106272473s FAIL (2025-08-17 04:22:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== .932058439s) [seg_pos={0 0}, upload=211221]: Done begin upload segment. Aug 17 04:21:34.973043873 (202h55m56.932141128s) [noise=true , address=77.90.31.12:4619 , piece_id=ZV4IQRGTWYICYQYP4CIL7BZ7LAQUKX5W2GG2ODDKN2K7K6OLTT5A, seg_pos={0 0}, upload=211221]: Uploading piece... Aug 17 04:21:34.973578324 (202h55m56.932675569s) [noise=true , address=104.36.82.59:4335 , piece_id=4TU7EBNQDH44I67IP3BLYBVTWLLQT7OXJFS5NXSWX3U4KVMO2K7A, seg_pos={0 0}, upload=211221]: Uploading piece... Aug 17 04:21:34.974003025 (202h55m56.933100276s) [noise=true , address=79.127.205.234:4595 , piece_id=GNJUDAQQ6CLWDAB6VLMLLGKRL3XG7N3RH6TDGRXAJH2TN3DUBIVA, seg_pos={0 0}, upload=211221]: Uploading piece... Aug 17 04:21:34.974144349 (202h55m56.933241602s) [noise=true , address=77.90.31.10:4544 , piece_id=JSNUWN55IRTNBM6E5JLHM7AGZJB2IGBDSWOOCZXLZMFYOGWS6GNA, seg_pos={0 0}, upload=211221]: Uploading piece... Aug 17 04:21:34.974334245 (202h55m56.933431502s) [noise=true , address=104.36.82.23:4465 , piece_id=KH7QO5VUI5BJ7O4AAKDIMYHXCTE2MEPQFL2SKLSBVIJDCOI5VAZA, seg_pos={0 0}, upload=211221]: Uploading piece... Aug 17 04:21:34.974547613 (202h55m56.933644861s) [noise=true , address=104.36.82.5:4855 , piece_id=WRAI37UHGY7A2FDWTNFKZDOLKPTHOAGUJG6FJBXSO3TA6HVWAYWA, seg_pos={0 0}, upload=211221]: Uploading piece... Aug 17 04:21:34.974656868 (202h55m56.933754123s) [noise=true , address=77.90.31.18:4838 , piece_id=54GWVW67MHGMJWMA2WIBNV5FWCY6XFJSKX3POYNTIOM7JETZUO6A, seg_pos={0 0}, upload=211221]: Uploading piece... Aug 17 04:21:34.974921103 (202h55m56.934018349s) [noise=true , address=79.127.220.101:4068 , piece_id=RCV74NEFXPZQHJD5VVAWNR6NVQJNPPTHIGCHLVOIVXZUIN4JRAKA, seg_pos={0 0}, upload=211221]: Uploading piece... Aug 17 04:21:34.975011441 (202h55m56.934108696s) [noise=true , address=104.36.82.9:4025 , piece_id=ZWEK6RGMHXDPD25HMYJ5IYINQNK2E2LXMB3SILECG4MU7FVYXNCA, seg_pos={0 0}, upload=211221]: Uploading piece... Aug 17 04:21:34.975298015 (202h55m56.934395264s) [noise=true , address=77.90.31.11:4559 , piece_id=PV77Z4ECT4IGCDJSY5LP76NM74UQTO4ZHWGLMIZBH355IBQMJZOQ, seg_pos={0 0}, upload=211221]: Uploading piece... Aug 17 04:21:34.975377642 (202h55m56.934474898s) [noise=true , address=66.23.193.175:4553 , piece_id=EVY6BGBOG5KI34GONABOSJQFQ53FJ7QKH5JF5A6IX3HX7PXI23SA, seg_pos={0 0}, upload=211221]: Uploading piece... Aug 17 04:21:34.975684906 (202h55m56.934782156s) [noise=true , address=77.90.31.19:4869 , piece_id=UWKEKFZSLXLGWVCAKHVEB7LQHKTDE5ILXIHVOEXZGJQBOYI7S2TQ, seg_pos={0 0}, upload=211221]: Uploading piece... Aug 17 04:21:34.975736971 (202h55m56.934834227s) [noise=true , address=79.127.201.214:4320 , piece_id=S5UGI7EYIGM6ROXVU7PQQPHPZYJ2FTZFUKU7EGNENVPOLAC6O74Q, seg_pos={0 0}, upload=211221]: Uploading piece... Aug 17 04:21:34.976110815 (202h55m56.93520807s) [noise=true , address=104.36.82.19:4414 , piece_id=4UJTDRFAQ7AA5EJLNGHC2R56GAWCGL7OLURNXJFWJRRN4F5O4CMA, seg_pos={0 0}, upload=211221]: Uploading piece... Aug 17 04:21:34.976136107 (202h55m56.935233355s) [noise=true , address=104.36.82.60:4347 , piece_id=I3KDP3YJTUK4I7OJL2F2GLOAPQGUTINQEILYGYQTURBTU5JIWQOA, seg_pos={0 0}, upload=211221]: Uploading piece... Aug 17 04:21:34.976568000 (202h55m56.935665248s) [noise=true , address=79.127.219.47:4060 , piece_id=S63NA44PTHUHMLDAIRH7GUXFP2U6SJNIBQ6YVG34AOUV5IQX3ELQ, seg_pos={0 0}, upload=211221]: Uploading piece... Aug 17 04:21:34.976589989 (202h55m56.935687244s) [noise=true , address=104.36.82.44:4891 , piece_id=BH6BVR64YN7LMXLHWPHCV6QHNAU4SJP6FQ4HDSSF6ARHKHYZY5CQ, seg_pos={0 0}, upload=211221]: Uploading piece... Aug 17 04:21:34.976964447 (202h55m56.936061699s) [noise=true , address=77.90.31.15:4708 , piece_id=3YBIWEO6OAOXRTPVW4M3QW7MLI53A6X44FN5SITZYWYLPQ4WDPFQ, seg_pos={0 0}, upload=211221]: Uploading piece... Aug 17 04:21:35.532377604 (202h55m57.491474851s) [upload=211221]: Next returned nil segment Aug 17 04:21:35.532400728 (202h55m57.491497975s) [upload=211221]: Waiting for error group managing segments... Aug 17 04:21:35.675470634 (202h55m57.634567867s) [noise=true , address=77.90.31.11:4559 , piece_id=PV77Z4ECT4IGCDJSY5LP76NM74UQTO4ZHWGLMIZBH355IBQMJZOQ, seg_pos={0 0}, upload=211221]: Done uploading piece. err: <nil> Aug 17 04:21:35.678289855 (202h55m57.637387092s) [noise=true , address=77.90.31.18:4838 , piece_id=54GWVW67MHGMJWMA2WIBNV5FWCY6XFJSKX3POYNTIOM7JETZUO6A, seg_pos={0 0}, upload=211221]: Done uploading piece. err: <nil> Aug 17 04:21:35.735667520 (202h55m57.694764769s) [noise=true , address=79.127.219.47:4060 , piece_id=S63NA44PTHUHMLDAIRH7GUXFP2U6SJNIBQ6YVG34AOUV5IQX3ELQ, seg_pos={0 0}, upload=211221]: Done uploading piece. err: <nil> Aug 17 04:21:35.743109761 (202h55m57.702207009s) [noise=true , address=77.90.31.10:4544 , piece_id=JSNUWN55IRTNBM6E5JLHM7AGZJB2IGBDSWOOCZXLZMFYOGWS6GNA, seg_pos={0 0}, upload=211221]: Done uploading piece. err: <nil> Aug 17 04:21:35.753401032 (202h55m57.712498369s) [noise=true , address=77.90.31.19:4869 , piece_id=UWKEKFZSLXLGWVCAKHVEB7LQHKTDE5ILXIHVOEXZGJQBOYI7S2TQ, seg_pos={0 0}, upload=211221]: Done uploading piece. err: <nil> Aug 17 04:21:35.825277722 (202h55m57.78437496s) [noise=true , address=77.90.31.15:4708 , piece_id=3YBIWEO6OAOXRTPVW4M3QW7MLI53A6X44FN5SITZYWYLPQ4WDPFQ, seg_pos={0 0}, upload=211221]: Done uploading piece. err: <nil> Aug 17 04:21:35.837368249 (202h55m57.796465486s) [noise=true , address=77.90.31.12:4619 , piece_id=ZV4IQRGTWYICYQYP4CIL7BZ7LAQUKX5W2GG2ODDKN2K7K6OLTT5A, seg_pos={0 0}, upload=211221]: Done uploading piece. err: <nil> Aug 17 04:21:35.862653102 (202h55m57.821750347s) [noise=true , address=79.127.220.101:4068 , piece_id=RCV74NEFXPZQHJD5VVAWNR6NVQJNPPTHIGCHLVOIVXZUIN4JRAKA, seg_pos={0 0}, upload=211221]: Done uploading piece. err: <nil> Aug 17 04:21:35.894542140 (202h55m57.85363939s) [noise=true , address=79.127.201.214:4320 , piece_id=S5UGI7EYIGM6ROXVU7PQQPHPZYJ2FTZFUKU7EGNENVPOLAC6O74Q, seg_pos={0 0}, upload=211221]: Done uploading piece. err: <nil> Aug 17 04:21:36.292948967 (202h55m58.252046199s) [noise=true , address=66.23.193.175:4553 , piece_id=EVY6BGBOG5KI34GONABOSJQFQ53FJ7QKH5JF5A6IX3HX7PXI23SA, seg_pos={0 0}, upload=211221]: Done uploading piece. err: <nil> Aug 17 04:21:36.296701484 (202h55m58.255798723s) [noise=true , address=79.127.205.234:4595 , piece_id=GNJUDAQQ6CLWDAB6VLMLLGKRL3XG7N3RH6TDGRXAJH2TN3DUBIVA, seg_pos={0 0}, upload=211221]: Done uploading piece. err: <nil> Aug 17 04:22:34.486800356 (202h56m56.445897604s) [noise=true , address=104.36.82.9:4025 , piece_id=ZWEK6RGMHXDPD25HMYJ5IYINQNK2E2LXMB3SILECG4MU7FVYXNCA, seg_pos={0 0}, upload=211221]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: rpc: dial tcp 104.36.82.9:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:34.487077504 (202h56m56.446188191s) [noise=true , address=104.36.82.60:4347 , piece_id=I3KDP3YJTUK4I7OJL2F2GLOAPQGUTINQEILYGYQTURBTU5JIWQOA, seg_pos={0 0}, upload=211221]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: rpc: dial tcp 104.36.82.60:4347: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:34.487172881 (202h56m56.446270129s) [noise=true , address=104.36.82.19:4414 , piece_id=4UJTDRFAQ7AA5EJLNGHC2R56GAWCGL7OLURNXJFWJRRN4F5O4CMA, seg_pos={0 0}, upload=211221]: Done uploading piece. err: ecclient: upload failed (node:1R6GJNS7bXfsDLsp3CE9qcvTbcT44QmgvSFzVbzpukyVYsezRQ, address:104.36.82.19:4414): protocol: rpc: dial tcp 104.36.82.19:4414: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:34.487231581 (202h56m56.44632883s) [noise=true , address=104.36.82.5:4855 , piece_id=WRAI37UHGY7A2FDWTNFKZDOLKPTHOAGUJG6FJBXSO3TA6HVWAYWA, seg_pos={0 0}, upload=211221]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 139.59.144.22:56554->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:34.487284710 (202h56m56.446381958s) [noise=true , address=104.36.82.23:4465 , piece_id=KH7QO5VUI5BJ7O4AAKDIMYHXCTE2MEPQFL2SKLSBVIJDCOI5VAZA, seg_pos={0 0}, upload=211221]: Done uploading piece. err: ecclient: upload failed (node:1Mt7mGfsy6qpxmZTiKcFavb3o8zYLJdk3tXDtGReLLyBkgu2jw, address:104.36.82.23:4465): protocol: rpc: dial tcp 104.36.82.23:4465: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:34.487316998 (202h56m56.446414244s) [noise=true , address=104.36.82.59:4335 , piece_id=4TU7EBNQDH44I67IP3BLYBVTWLLQT7OXJFS5NXSWX3U4KVMO2K7A, seg_pos={0 0}, upload=211221]: Done uploading piece. err: ecclient: upload failed (node:12gFW6PBz447rjBeSsnrdskgBcCHrqvB1SDCcLBzHU8G972fUm2, address:104.36.82.59:4335): protocol: write tcp 139.59.144.22:51532->104.36.82.59:4335: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:34.487335540 (202h56m56.446432789s) [noise=true , address=104.36.82.44:4891 , piece_id=BH6BVR64YN7LMXLHWPHCV6QHNAU4SJP6FQ4HDSSF6ARHKHYZY5CQ, seg_pos={0 0}, upload=211221]: Done uploading piece. err: ecclient: upload failed (node:1RHLM4MJmtKoHsUnsqccy1ehyD7zVSn6gnJNp7yJJmZhTJqPkU, address:104.36.82.44:4891): protocol: rpc: dial tcp 104.36.82.44:4891: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:34.487356962 (202h56m56.446454216s) [seg_pos={0 0}, upload=211221]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Aug 17 04:22:34.590918346 (202h56m56.550015592s) [upload=211221]: Done uploading segments... Aug 17 04:22:34.590966047 (202h56m56.550063293s) [upload=211221]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 1m0.106287772s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.000531263s FAIL (2025-08-17 04:22:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000544165s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.104077978s FAIL (2025-08-17 04:21:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 04:21:29.261142740 (202h55m51.220239971s) [upload=211220]: Starting upload Aug 17 04:21:29.261204488 (202h55m51.220301733s) [upload=211220]: Done starting upload Aug 17 04:21:29.261306583 (202h55m51.220403835s) [upload=211220]: Uploading segments... Aug 17 04:21:29.261450754 (202h55m51.220547985s) [upload=211220]: Got next segment. Inline: false Aug 17 04:21:29.261485617 (202h55m51.220582862s) [upload=211220]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:21:29.261489944 (202h55m51.220587175s) [upload=211220]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:21:29.478234539 (202h55m51.437331776s) [seg_pos={0 0}, upload=211220]: Begin upload segment... Aug 17 04:21:29.478301885 (202h55m51.43739912s) [seg_pos={0 0}, upload=211220]: Done begin upload segment. Aug 17 04:21:29.478352046 (202h55m51.437449298s) [noise=true , address=45.91.116.4:4938 , piece_id=KWXS2ZCR2LHS4TI2FHSJ5QGZSCWSW362ZB5XCKUVOSBGDFQARKJQ, seg_pos={0 0}, upload=211220]: Uploading piece... Aug 17 04:21:29.478783997 (202h55m51.437881245s) [noise=true , address=104.36.82.15:4257 , piece_id=WI4JRYYYLEZXW5KMDN4I3AQ57IFGE2JK3EU3D34RCZALQF26YXVQ, seg_pos={0 0}, upload=211220]: Uploading piece... Aug 17 04:21:29.479126712 (202h55m51.438223963s) [noise=true , address=104.36.82.12:4150 , piece_id=5IEX2XAMGNCLTG4SMVRYVZJG5BEWIMZA7ZRHUDSNT6VJHY55XECQ, seg_pos={0 0}, upload=211220]: Uploading piece... Aug 17 04:21:29.479164199 (202h55m51.438261483s) [noise=true , address=104.36.82.8:4971 , piece_id=I7ROKIHQ2FTR2OWADVNAP5NO7BOPDBKFB7PYUGF423ZTWRUJ4JOQ, seg_pos={0 0}, upload=211220]: Uploading piece... Aug 17 04:21:29.479485512 (202h55m51.43858276s) [noise=true , address=77.90.31.19:4841 , piece_id=EY7ZCKC3VBLCMH633JBZFRQNGIAJJDXUL54GYN4VVMFZCV7ITX3Q, seg_pos={0 0}, upload=211220]: Uploading piece... Aug 17 04:21:29.479536614 (202h55m51.438633863s) [noise=true , address=45.91.116.4:4940 , piece_id=6UP6ZHMXMOXPWBSVU4UCVNSVSMQU7OPCWZHTO3CQM2YEPGJFCTUQ, seg_pos={0 0}, upload=211220]: Uploading piece... Aug 17 04:21:29.479778991 (202h55m51.438876237s) [noise=true , address=45.91.116.7:4009 , piece_id=6DIW6OCINK5P5SZCCG5FYS7XRTMKU7JICPP2L4XNIUOPKU63IKNA, seg_pos={0 0}, upload=211220]: Uploading piece... Aug 17 04:21:29.479789666 (202h55m51.438886911s) [noise=true , address=104.36.82.10:4078 , piece_id=2PZKDXHBDA5HISBB6MOO7UGBKZD6VPOMD5VPXXSGP7Y4OD42YOOA, seg_pos={0 0}, upload=211220]: Uploading piece... Aug 17 04:21:29.480024255 (202h55m51.4391215s) [noise=true , address=104.36.82.16:4295 , piece_id=IZZK2TVGTF3SD45ZS4NXRYXCLE3HQJIMIZOTVNZB5PP6ODGDOM5A, seg_pos={0 0}, upload=211220]: Uploading piece... Aug 17 04:21:29.480080109 (202h55m51.439177357s) [noise=true , address=45.91.116.9:4261 , piece_id=QV6ZBGHZBARGGRVO6MEBJDIDXWDUKQV63XKARFK5MJFYWJGNYDOQ, seg_pos={0 0}, upload=211220]: Uploading piece... Aug 17 04:21:29.480269517 (202h55m51.439366762s) [noise=true , address=77.90.31.11:4554 , piece_id=LZ76PKSW5SIRECBEBL27TRL5CS46AXF6V2GTJXY3B5OAS5YIDCVQ, seg_pos={0 0}, upload=211220]: Uploading piece... Aug 17 04:21:29.480484691 (202h55m51.439581937s) [noise=true , address=104.36.82.2:4732 , piece_id=N4WMXYI2OKSWZJD7VR5ZUZ5L7KXSIM4Z2X6QN7YR7FGWT7XWUYYA, seg_pos={0 0}, upload=211220]: Uploading piece... Aug 17 04:21:29.480716562 (202h55m51.439813813s) [noise=true , address=77.90.31.18:4807 , piece_id=JDMKB546IQCCOEX2NKBOWXMND54I6S4VHZEUZSWK7SFIBP4SPCVA, seg_pos={0 0}, upload=211220]: Uploading piece... Aug 17 04:21:29.481011311 (202h55m51.440108557s) [noise=true , address=45.91.116.5:4297 , piece_id=WBOGPMDE776MSCEAUFX6ULI75A72JNTDQAZLNLVMPF6WGDHSZOGQ, seg_pos={0 0}, upload=211220]: Uploading piece... Aug 17 04:21:29.481292512 (202h55m51.440389746s) [noise=true , address=104.36.82.3:4767 , piece_id=SMGJA35C5TYESKN4VCHGPDEB3EVD6FTGQULC2M5APRPT3KHQKRIA, seg_pos={0 0}, upload=211220]: Uploading piece... Aug 17 04:21:29.482893493 (202h55m51.441990725s) [noise=true , address=45.91.116.5:4298 , piece_id=Z7A3K7ABDXVYGR74GYEMU7A2WM2LJ32VCOGDPT56GCBEK5AWZTLA, seg_pos={0 0}, upload=211220]: Uploading piece... Aug 17 04:21:30.090484072 (202h55m52.049581309s) [upload=211220]: Next returned nil segment Aug 17 04:21:30.090501561 (202h55m52.049598806s) [upload=211220]: Waiting for error group managing segments... Aug 17 04:21:30.229374476 (202h55m52.188471723s) [noise=true , address=45.91.116.9:4261 , piece_id=QV6ZBGHZBARGGRVO6MEBJDIDXWDUKQV63XKARFK5MJFYWJGNYDOQ, seg_pos={0 0}, upload=211220]: Done uploading piece. err: <nil> Aug 17 04:21:30.233385693 (202h55m52.192482932s) [noise=true , address=45.91.116.5:4298 , piece_id=Z7A3K7ABDXVYGR74GYEMU7A2WM2LJ32VCOGDPT56GCBEK5AWZTLA, seg_pos={0 0}, upload=211220]: Done uploading piece. err: <nil> Aug 17 04:21:30.238173358 (202h55m52.197270604s) [noise=true , address=45.91.116.4:4940 , piece_id=6UP6ZHMXMOXPWBSVU4UCVNSVSMQU7OPCWZHTO3CQM2YEPGJFCTUQ, seg_pos={0 0}, upload=211220]: Done uploading piece. err: <nil> Aug 17 04:21:30.253224382 (202h55m52.212321621s) [noise=true , address=45.91.116.5:4297 , piece_id=WBOGPMDE776MSCEAUFX6ULI75A72JNTDQAZLNLVMPF6WGDHSZOGQ, seg_pos={0 0}, upload=211220]: Done uploading piece. err: <nil> Aug 17 04:21:30.259699945 (202h55m52.21879718s) [noise=true , address=45.91.116.4:4938 , piece_id=KWXS2ZCR2LHS4TI2FHSJ5QGZSCWSW362ZB5XCKUVOSBGDFQARKJQ, seg_pos={0 0}, upload=211220]: Done uploading piece. err: <nil> Aug 17 04:21:30.328117483 (202h55m52.287214721s) [noise=true , address=77.90.31.11:4554 , piece_id=LZ76PKSW5SIRECBEBL27TRL5CS46AXF6V2GTJXY3B5OAS5YIDCVQ, seg_pos={0 0}, upload=211220]: Done uploading piece. err: <nil> Aug 17 04:21:30.333693663 (202h55m52.292790901s) [noise=true , address=77.90.31.18:4807 , piece_id=JDMKB546IQCCOEX2NKBOWXMND54I6S4VHZEUZSWK7SFIBP4SPCVA, seg_pos={0 0}, upload=211220]: Done uploading piece. err: <nil> Aug 17 04:21:31.795037264 (202h55m53.754134496s) [noise=true , address=45.91.116.7:4009 , piece_id=6DIW6OCINK5P5SZCCG5FYS7XRTMKU7JICPP2L4XNIUOPKU63IKNA, seg_pos={0 0}, upload=211220]: Done uploading piece. err: <nil> Aug 17 04:21:31.904109662 (202h55m53.863206918s) [noise=true , address=77.90.31.19:4841 , piece_id=EY7ZCKC3VBLCMH633JBZFRQNGIAJJDXUL54GYN4VVMFZCV7ITX3Q, seg_pos={0 0}, upload=211220]: Done uploading piece. err: <nil> Aug 17 04:21:58.984753850 (202h56m20.943851087s) [noise=true , address=104.36.82.12:4150 , piece_id=5IEX2XAMGNCLTG4SMVRYVZJG5BEWIMZA7ZRHUDSNT6VJHY55XECQ, seg_pos={0 0}, upload=211220]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: rpc: dial tcp 104.36.82.12:4150: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.984967691 (202h56m20.944064937s) [noise=true , address=104.36.82.16:4295 , piece_id=IZZK2TVGTF3SD45ZS4NXRYXCLE3HQJIMIZOTVNZB5PP6ODGDOM5A, seg_pos={0 0}, upload=211220]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: rpc: dial tcp 104.36.82.16:4295: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.985067848 (202h56m20.944165101s) [noise=true , address=104.36.82.15:4257 , piece_id=WI4JRYYYLEZXW5KMDN4I3AQ57IFGE2JK3EU3D34RCZALQF26YXVQ, seg_pos={0 0}, upload=211220]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.985170538 (202h56m20.944267793s) [noise=true , address=104.36.82.8:4971 , piece_id=I7ROKIHQ2FTR2OWADVNAP5NO7BOPDBKFB7PYUGF423ZTWRUJ4JOQ, seg_pos={0 0}, upload=211220]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: rpc: dial tcp 104.36.82.8:4971: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.985246018 (202h56m20.944343257s) [noise=true , address=104.36.82.2:4732 , piece_id=N4WMXYI2OKSWZJD7VR5ZUZ5L7KXSIM4Z2X6QN7YR7FGWT7XWUYYA, seg_pos={0 0}, upload=211220]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.985357905 (202h56m20.944455152s) [noise=true , address=104.36.82.3:4767 , piece_id=SMGJA35C5TYESKN4VCHGPDEB3EVD6FTGQULC2M5APRPT3KHQKRIA, seg_pos={0 0}, upload=211220]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: rpc: dial tcp 104.36.82.3:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.985411500 (202h56m20.944508738s) [noise=true , address=104.36.82.10:4078 , piece_id=2PZKDXHBDA5HISBB6MOO7UGBKZD6VPOMD5VPXXSGP7Y4OD42YOOA, seg_pos={0 0}, upload=211220]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 139.59.144.22:54994->104.36.82.10:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.985458527 (202h56m20.944555772s) [seg_pos={0 0}, upload=211220]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 04:21:59.087003832 (202h56m21.046101069s) [upload=211220]: Done uploading segments... Aug 17 04:21:59.087031233 (202h56m21.046128478s) [upload=211220]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.104104793s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X....................................................XX] duration: 1m0.104211098s FAIL (2025-08-17 04:21:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 04:20:58.573202465 (202h55m20.532299711s) [upload=211209]: Starting upload Aug 17 04:20:58.573273336 (202h55m20.532370582s) [upload=211209]: Done starting upload Aug 17 04:20:58.573544571 (202h55m20.532641818s) [upload=211209]: Uploading segments... Aug 17 04:20:58.573639927 (202h55m20.532737172s) [upload=211209]: Got next segment. Inline: false Aug 17 04:20:58.573680839 (202h55m20.532778084s) [upload=211209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:20:58.573686999 (202h55m20.532784245s) [upload=211209]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:20:58.774489366 (202h55m20.733586613s) [seg_pos={0 0}, upload=211209]: Begin upload segment... Aug 17 04:20:58.774551179 (202h55m20.733648427s) [seg_pos={0 0}, upload=211209]: Done begin upload segment. Aug 17 04:20:58.774580416 (202h55m20.733677664s) [noise=true , address=104.36.82.39:4740 , piece_id=XDCMKR6QF4MS2EM4YPFWXXVZZA67K4GOA6BI2YR6O4NNWXXWRZYA, seg_pos={0 0}, upload=211209]: Uploading piece... Aug 17 04:20:58.775053117 (202h55m20.734150365s) [noise=true , address=45.91.116.10:4103 , piece_id=EUTS5OK3CU3WLEBSFT3PCLUOMZSPAYIU3TFBYGYBJLLCRWC2AGDQ, seg_pos={0 0}, upload=211209]: Uploading piece... Aug 17 04:20:58.775399671 (202h55m20.734496916s) [noise=true , address=45.91.116.9:4272 , piece_id=3ZUM7N74JKRA5BCKOZCTFFOMG7FBMAHZY3G5AA4ASBVRE6ISL2SA, seg_pos={0 0}, upload=211209]: Uploading piece... Aug 17 04:20:58.775724152 (202h55m20.734821403s) [noise=true , address=66.23.193.175:4559 , piece_id=JGNO6BWC7G5WO6VUSZJK2IRBEKPHKTHYTCDNZBSEEAD4RUBFXARQ, seg_pos={0 0}, upload=211209]: Uploading piece... Aug 17 04:20:58.775968750 (202h55m20.735065995s) [noise=true , address=79.127.201.215:4353 , piece_id=4WBC5XOJNS76QG2JWDD2MOYLMH4SLZAXUQREBNDD2ZG2HT64CG6A, seg_pos={0 0}, upload=211209]: Uploading piece... Aug 17 04:20:58.776710863 (202h55m20.735808115s) [noise=true , address=104.36.82.44:4893 , piece_id=AT45IXYQ3APBKJW67V2S76ZJX6RKLXJX4U6REZVLJYUXGN3BH4MA, seg_pos={0 0}, upload=211209]: Uploading piece... Aug 17 04:20:58.776799248 (202h55m20.735896497s) [noise=true , address=104.36.82.60:4356 , piece_id=LZO4NPVZWSUCZOO5QRQMGMSU2N22BAS6F7MGSQQ6DEJS5OJP3Z7A, seg_pos={0 0}, upload=211209]: Uploading piece... Aug 17 04:20:58.777553021 (202h55m20.736650276s) [noise=true , address=104.36.82.23:4466 , piece_id=6D7VACU7N25MVUKEXFOYUDQHO2WZ6JOJFVZKY4SBBK7USKTSEOGA, seg_pos={0 0}, upload=211209]: Uploading piece... Aug 17 04:20:58.778154591 (202h55m20.737251844s) [noise=true , address=77.90.31.13:4660 , piece_id=UHGNJNU333G6H3JUVSEEPEHLTQGBL2NMZZB34YBUZOUHTQO4R4SA, seg_pos={0 0}, upload=211209]: Uploading piece... Aug 17 04:20:58.778618135 (202h55m20.737715387s) [noise=true , address=77.90.31.11:4580 , piece_id=W5AODOLOT6RFWUB3D72M6B2Z5E53EF7AJXVMMHJA7WTMKM37PJSQ, seg_pos={0 0}, upload=211209]: Uploading piece... Aug 17 04:20:58.779017510 (202h55m20.738114763s) [noise=true , address=77.90.31.10:4527 , piece_id=22LG7JXIBICGAUDGFBSG4UG5YI7NZY7N3GQIO446D4V73JWEKEFA, seg_pos={0 0}, upload=211209]: Uploading piece... Aug 17 04:20:58.779412007 (202h55m20.73850926s) [noise=true , address=79.127.201.215:4356 , piece_id=Z2ENVJ5EQWL5G2IAK7M7Z26LHRGWZN52L4LTGK5ACEEEJASQ5AZQ, seg_pos={0 0}, upload=211209]: Uploading piece... Aug 17 04:20:58.779793082 (202h55m20.738890335s) [noise=true , address=104.36.82.19:4404 , piece_id=EXV7ANJGWEVL7YTGSM7H73EAG7XQYLSJXWTFA4MUISWXZCURJWAA, seg_pos={0 0}, upload=211209]: Uploading piece... Aug 17 04:20:58.780164311 (202h55m20.739261565s) [noise=true , address=66.23.193.175:4556 , piece_id=KYJU3F6IFCQKEAXI5Z3KYPOLPA5YP6C7AIIGU3A3QZ4GUCX5SBIA, seg_pos={0 0}, upload=211209]: Uploading piece... Aug 17 04:20:58.780372148 (202h55m20.739469392s) [noise=true , address=104.36.82.5:4869 , piece_id=4UCBUHRGGFVHAZHRSZBVSL2FK5NVEZESRLP5R22HP2GWR7CXWR4A, seg_pos={0 0}, upload=211209]: Uploading piece... Aug 17 04:20:58.780828692 (202h55m20.739925946s) [noise=true , address=104.36.82.2:4760 , piece_id=K37OBBWH5YU5OQNSJEQXKPESJXYPCMOWX5HZ3FLKRBK2C4SJJKXA, seg_pos={0 0}, upload=211209]: Uploading piece... Aug 17 04:20:58.781223246 (202h55m20.740320494s) [noise=true , address=45.91.116.7:4041 , piece_id=3USJQEMA5PTCMP757DQY22LPIWSEUVM4KLPK6QMUKBKSA53JF7PQ, seg_pos={0 0}, upload=211209]: Uploading piece... Aug 17 04:20:58.781587620 (202h55m20.740684871s) [noise=true , address=104.36.82.59:4327 , piece_id=QEEM4DUMR5OVCK4AC5HKWINGOHPYGRRU7UY2ZLBGBXEKHIKZAYQQ, seg_pos={0 0}, upload=211209]: Uploading piece... Aug 17 04:20:59.339555783 (202h55m21.298653023s) [upload=211209]: Next returned nil segment Aug 17 04:20:59.339574855 (202h55m21.2986721s) [upload=211209]: Waiting for error group managing segments... Aug 17 04:20:59.478419780 (202h55m21.437517027s) [noise=true , address=45.91.116.10:4103 , piece_id=EUTS5OK3CU3WLEBSFT3PCLUOMZSPAYIU3TFBYGYBJLLCRWC2AGDQ, seg_pos={0 0}, upload=211209]: Done uploading piece. err: <nil> Aug 17 04:20:59.492980781 (202h55m21.452078019s) [noise=true , address=45.91.116.7:4041 , piece_id=3USJQEMA5PTCMP757DQY22LPIWSEUVM4KLPK6QMUKBKSA53JF7PQ, seg_pos={0 0}, upload=211209]: Done uploading piece. err: <nil> Aug 17 04:20:59.497529167 (202h55m21.456626398s) [noise=true , address=45.91.116.9:4272 , piece_id=3ZUM7N74JKRA5BCKOZCTFFOMG7FBMAHZY3G5AA4ASBVRE6ISL2SA, seg_pos={0 0}, upload=211209]: Done uploading piece. err: <nil> Aug 17 04:20:59.544004344 (202h55m21.50310158s) [noise=true , address=77.90.31.13:4660 , piece_id=UHGNJNU333G6H3JUVSEEPEHLTQGBL2NMZZB34YBUZOUHTQO4R4SA, seg_pos={0 0}, upload=211209]: Done uploading piece. err: <nil> Aug 17 04:20:59.585439600 (202h55m21.544536851s) [noise=true , address=77.90.31.10:4527 , piece_id=22LG7JXIBICGAUDGFBSG4UG5YI7NZY7N3GQIO446D4V73JWEKEFA, seg_pos={0 0}, upload=211209]: Done uploading piece. err: <nil> Aug 17 04:20:59.632020461 (202h55m21.591117699s) [noise=true , address=77.90.31.11:4580 , piece_id=W5AODOLOT6RFWUB3D72M6B2Z5E53EF7AJXVMMHJA7WTMKM37PJSQ, seg_pos={0 0}, upload=211209]: Done uploading piece. err: <nil> Aug 17 04:20:59.749640667 (202h55m21.708737899s) [noise=true , address=79.127.201.215:4356 , piece_id=Z2ENVJ5EQWL5G2IAK7M7Z26LHRGWZN52L4LTGK5ACEEEJASQ5AZQ, seg_pos={0 0}, upload=211209]: Done uploading piece. err: <nil> Aug 17 04:20:59.823902225 (202h55m21.782999474s) [noise=true , address=79.127.201.215:4353 , piece_id=4WBC5XOJNS76QG2JWDD2MOYLMH4SLZAXUQREBNDD2ZG2HT64CG6A, seg_pos={0 0}, upload=211209]: Done uploading piece. err: <nil> Aug 17 04:21:00.019834825 (202h55m21.978932064s) [noise=true , address=66.23.193.175:4559 , piece_id=JGNO6BWC7G5WO6VUSZJK2IRBEKPHKTHYTCDNZBSEEAD4RUBFXARQ, seg_pos={0 0}, upload=211209]: Done uploading piece. err: <nil> Aug 17 04:21:00.048254439 (202h55m22.007351684s) [noise=true , address=66.23.193.175:4556 , piece_id=KYJU3F6IFCQKEAXI5Z3KYPOLPA5YP6C7AIIGU3A3QZ4GUCX5SBIA, seg_pos={0 0}, upload=211209]: Done uploading piece. err: <nil> Aug 17 04:21:58.297503606 (202h56m20.256600858s) [noise=true , address=104.36.82.60:4356 , piece_id=LZO4NPVZWSUCZOO5QRQMGMSU2N22BAS6F7MGSQQ6DEJS5OJP3Z7A, seg_pos={0 0}, upload=211209]: Done uploading piece. err: ecclient: upload failed (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1, address:104.36.82.60:4356): protocol: rpc: dial tcp 104.36.82.60:4356: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.297681142 (202h56m20.256778392s) [noise=true , address=104.36.82.23:4466 , piece_id=6D7VACU7N25MVUKEXFOYUDQHO2WZ6JOJFVZKY4SBBK7USKTSEOGA, seg_pos={0 0}, upload=211209]: Done uploading piece. err: ecclient: upload failed (node:12rkNnk9RUhKBJ3EsakqMRb3wYwrhRLhB4ZYbrdfKPDk8erRJpP, address:104.36.82.23:4466): protocol: rpc: dial tcp 104.36.82.23:4466: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.297763357 (202h56m20.256860588s) [noise=true , address=104.36.82.59:4327 , piece_id=QEEM4DUMR5OVCK4AC5HKWINGOHPYGRRU7UY2ZLBGBXEKHIKZAYQQ, seg_pos={0 0}, upload=211209]: Done uploading piece. err: ecclient: upload failed (node:1V4C6AWGbyYQWfinoSj7iKovHeTJ9hqypyC5U9rZ8pveXBBfCY, address:104.36.82.59:4327): protocol: rpc: dial tcp 104.36.82.59:4327: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.297957612 (202h56m20.257054861s) [noise=true , address=104.36.82.44:4893 , piece_id=AT45IXYQ3APBKJW67V2S76ZJX6RKLXJX4U6REZVLJYUXGN3BH4MA, seg_pos={0 0}, upload=211209]: Done uploading piece. err: ecclient: upload failed (node:17VMFZ2T8jrSJVtchciJ9SnyrR6JstvcMoXrN5Q3zTD1aQU1oJ, address:104.36.82.44:4893): protocol: write tcp 139.59.144.22:41388->104.36.82.44:4893: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.298068585 (202h56m20.257165841s) [noise=true , address=104.36.82.19:4404 , piece_id=EXV7ANJGWEVL7YTGSM7H73EAG7XQYLSJXWTFA4MUISWXZCURJWAA, seg_pos={0 0}, upload=211209]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:104.36.82.19:4404): protocol: rpc: dial tcp 104.36.82.19:4404: i/o timeout; conn closed: op Aug 17 04:21:58.298371564 (202h56m20.257468812s) [noise=true , address=104.36.82.5:4869 , piece_id=4UCBUHRGGFVHAZHRSZBVSL2FK5NVEZESRLP5R22HP2GWR7CXWR4A, seg_pos={0 0}, upload=211209]: Done uploading piece. err: ecclient: upload failed (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr, address:104.36.82.5:4869): protocol: rpc: dial tcp 104.36.82.5:4869: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.298438262 (202h56m20.257535509s) [noise=true , address=104.36.82.2:4760 , piece_id=K37OBBWH5YU5OQNSJEQXKPESJXYPCMOWX5HZ3FLKRBK2C4SJJKXA, seg_pos={0 0}, upload=211209]: Done uploading piece. err: ecclient: upload failed (node:1pgY3KuDopCrLXxvbVeNowHD9qSh2Gz7qcgUR1GpsxFkuoHhzB, address:104.36.82.2:4760): protocol: rpc: dial tcp 104.36.82.2:4760: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.298488747 (202h56m20.257586003s) [noise=true , address=104.36.82.39:4740 , piece_id=XDCMKR6QF4MS2EM4YPFWXXVZZA67K4GOA6BI2YR6O4NNWXXWRZYA, seg_pos={0 0}, upload=211209]: Done uploading piece. err: ecclient: upload failed (node:1A2tCUpDAzrSzhUfBf6GMsZdpY6SPkRHWnqrWiiy7uNGfzcYbq, address:104.36.82.39:4740): protocol: write tcp 139.59.144.22:49906->104.36.82.39:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.298536033 (202h56m20.257633291s) [seg_pos={0 0}, upload=211209]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:21:58.399840587 (202h56m20.35893783s) [upload=211209]: Done uploading segments... Aug 17 04:21:58.399888999 (202h56m20.358986246s) [upload=211209]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.104240674s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 30.002007923s FAIL (2025-08-17 04:21:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.002023014s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.10226091s FAIL (2025-08-17 04:21:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 7) ======LOGS====== pos={0 0}, upload=211204]: Uploading piece... Aug 17 04:20:34.698913847 (202h54m56.658011113s) [noise=true , address=104.36.82.23:4458 , piece_id=HFGIKNJJVBDUENWFPNOCPRTBXF2IV7HCWKYYCFILGCHD7KHYWRJA, seg_pos={0 0}, upload=211204]: Uploading piece... Aug 17 04:20:34.698944996 (202h54m56.658042244s) [noise=true , address=104.36.82.3:4790 , piece_id=2QXJSMA4FMHVR2LBE4QRYMKRWO6H6MESP3W6KCAGGYFCD7LOZKRQ, seg_pos={0 0}, upload=211204]: Uploading piece... Aug 17 04:20:34.699287014 (202h54m56.658384261s) [noise=true , address=104.36.82.58:4286 , piece_id=JWFVQA4FLL3CG6VWEI6EHPBWDWBI7KJ5REVVSJ23HHCTH6HILTSA, seg_pos={0 0}, upload=211204]: Uploading piece... Aug 17 04:20:34.699304762 (202h54m56.658402007s) [noise=true , address=104.36.82.59:4321 , piece_id=QBHNUDQXWSWWHOSJOXJ5DGKSNLMWOZZ7B7Z5EACGXJWKDAA6ZLGA, seg_pos={0 0}, upload=211204]: Uploading piece... Aug 17 04:20:34.699604726 (202h54m56.658701971s) [noise=true , address=104.36.82.60:4352 , piece_id=CSG6QHVS3X4ZYNPLB7XTJVNHLOO25743ATYLF2PIIF22PNSFEEWQ, seg_pos={0 0}, upload=211204]: Uploading piece... Aug 17 04:20:34.699701084 (202h54m56.658798338s) [noise=true , address=77.90.31.14:4667 , piece_id=G7TDJU4KDFAZ4HEMWM25VQLIKTPLWOQRVIIT4C465EF4LWYBWBMA, seg_pos={0 0}, upload=211204]: Uploading piece... Aug 17 04:20:34.700023586 (202h54m56.659120841s) [noise=true , address=104.36.82.19:4417 , piece_id=55MOETZ27XOYPWIXOU7JSTJZXWLTYR5J6ZNDVRFKI74OSNRJMBNQ, seg_pos={0 0}, upload=211204]: Uploading piece... Aug 17 04:20:34.700069492 (202h54m56.659166744s) [noise=true , address=45.91.116.14:4210 , piece_id=KAGZT4R4MF3CMQECTQH2PYVKMUHQ3ZXTG46FJ5CBYE3WKNRXLLTA, seg_pos={0 0}, upload=211204]: Uploading piece... Aug 17 04:20:34.700403515 (202h54m56.659500762s) [noise=true , address=104.36.82.21:4507 , piece_id=M3V6L3F3QGA6FVMF6YJVQBYWJ7SQSL3CRDP3WFMCTD6GNQNN54SA, seg_pos={0 0}, upload=211204]: Uploading piece... Aug 17 04:20:34.700594952 (202h54m56.659692206s) [noise=true , address=66.23.193.175:4563 , piece_id=L6ITKC4U4R5K3JXPW3TVYIXVBYHRVNPKP74OK4NHV32QKDJINBUA, seg_pos={0 0}, upload=211204]: Uploading piece... Aug 17 04:20:34.700741285 (202h54m56.659838534s) [noise=true , address=104.36.82.35:4609 , piece_id=25IQQMRZH7LUC2PBORIR3TEE3C74CLKEZGTAAB2C3RI2375YKKKQ, seg_pos={0 0}, upload=211204]: Uploading piece... Aug 17 04:20:34.700943824 (202h54m56.660041069s) [noise=true , address=45.91.116.13:4166 , piece_id=DDF44DMUIB3Z3RJ4WXP3A3G6GV5OENBRQCROSSUCWRXJUOZM7KUA, seg_pos={0 0}, upload=211204]: Uploading piece... Aug 17 04:20:34.701045551 (202h54m56.660142802s) [noise=true , address=104.36.82.5:4857 , piece_id=7FM7H6OEK7ULAK472NM4P3VQ5UHV6VHV7BOIIDWFGMLG4C4F6JGQ, seg_pos={0 0}, upload=211204]: Uploading piece... Aug 17 04:20:34.701277409 (202h54m56.660374662s) [noise=true , address=104.36.82.34:4563 , piece_id=O6XS6ZW4TBSFXYJSFK6IZR3J5KPHFWJGV5YVSEKAI3NYLA5XAXRA, seg_pos={0 0}, upload=211204]: Uploading piece... Aug 17 04:20:35.275732075 (202h54m57.23482932s) [upload=211204]: Next returned nil segment Aug 17 04:20:35.275751863 (202h54m57.234849111s) [upload=211204]: Waiting for error group managing segments... Aug 17 04:20:35.429423675 (202h54m57.388520913s) [noise=true , address=45.91.116.14:4210 , piece_id=KAGZT4R4MF3CMQECTQH2PYVKMUHQ3ZXTG46FJ5CBYE3WKNRXLLTA, seg_pos={0 0}, upload=211204]: Done uploading piece. err: <nil> Aug 17 04:20:35.436755285 (202h54m57.395852523s) [noise=true , address=45.91.116.10:4102 , piece_id=WZSXFOO2TTAT74QAKR2MO4RY7U3XUCGXJHZYYNWGIJLLKZDVQDJA, seg_pos={0 0}, upload=211204]: Done uploading piece. err: <nil> Aug 17 04:20:35.457856297 (202h54m57.416953542s) [noise=true , address=79.127.219.47:4060 , piece_id=LVIV2B2YONBBWF5MQUKJMPC5U7QSWIW4IQPBWKTJ4VONKJ7Q7JBQ, seg_pos={0 0}, upload=211204]: Done uploading piece. err: <nil> Aug 17 04:20:35.571055279 (202h54m57.530152511s) [noise=true , address=45.91.116.13:4166 , piece_id=DDF44DMUIB3Z3RJ4WXP3A3G6GV5OENBRQCROSSUCWRXJUOZM7KUA, seg_pos={0 0}, upload=211204]: Done uploading piece. err: <nil> Aug 17 04:20:35.633694041 (202h54m57.59279128s) [noise=true , address=79.127.201.215:4354 , piece_id=LT46QR3FS4VTC666DDKZUUEFKMUIVYD42GMGKRBGFI7PA76JEW3A, seg_pos={0 0}, upload=211204]: Done uploading piece. err: <nil> Aug 17 04:20:35.683961835 (202h54m57.64305907s) [noise=true , address=77.90.31.14:4667 , piece_id=G7TDJU4KDFAZ4HEMWM25VQLIKTPLWOQRVIIT4C465EF4LWYBWBMA, seg_pos={0 0}, upload=211204]: Done uploading piece. err: <nil> Aug 17 04:20:36.059513553 (202h54m58.018610798s) [noise=true , address=66.23.193.175:4563 , piece_id=L6ITKC4U4R5K3JXPW3TVYIXVBYHRVNPKP74OK4NHV32QKDJINBUA, seg_pos={0 0}, upload=211204]: Done uploading piece. err: <nil> Aug 17 04:21:34.234565487 (202h55m56.193662741s) [noise=true , address=104.36.82.23:4458 , piece_id=HFGIKNJJVBDUENWFPNOCPRTBXF2IV7HCWKYYCFILGCHD7KHYWRJA, seg_pos={0 0}, upload=211204]: Done uploading piece. err: ecclient: upload failed (node:1TMPdLfCbmgi3GqDDg6AUvTCFPzbCCgwTAn9JJBrYJRnbkKtpb, address:104.36.82.23:4458): protocol: rpc: dial tcp 104.36.82.23:4458: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:34.234655875 (202h55m56.193753109s) [noise=true , address=104.36.82.21:4507 , piece_id=M3V6L3F3QGA6FVMF6YJVQBYWJ7SQSL3CRDP3WFMCTD6GNQNN54SA, seg_pos={0 0}, upload=211204]: Done uploading piece. err: ecclient: upload failed (node:1XW3SaTU4fPuKGNeLtt4H9wQXw5ADEcvzcpo7J53wUULsMT5Bv, address:104.36.82.21:4507): protocol: rpc: dial tcp 104.36.82.21:4507: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:34.234720162 (202h55m56.193817415s) [noise=true , address=104.36.82.19:4417 , piece_id=55MOETZ27XOYPWIXOU7JSTJZXWLTYR5J6ZNDVRFKI74OSNRJMBNQ, seg_pos={0 0}, upload=211204]: Done uploading piece. err: ecclient: upload failed (node:1TKc4R7hmma2b59RP5Qs5SEw8ha8FECxfJyTdz7mq6jao1JTCK, address:104.36.82.19:4417): protocol: rpc: dial tcp 104.36.82.19:4417: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:34.234773550 (202h55m56.193870801s) [noise=true , address=104.36.82.59:4321 , piece_id=QBHNUDQXWSWWHOSJOXJ5DGKSNLMWOZZ7B7Z5EACGXJWKDAA6ZLGA, seg_pos={0 0}, upload=211204]: Done uploading piece. err: ecclient: upload failed (node:1bN25eEEjxCWKJSr8qj7moDJxArgfsjzPTdBcLdfvoeyxiqPpc, address:104.36.82.59:4321): protocol: rpc: dial tcp 104.36.82.59:4321: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:34.234790846 (202h55m56.193888089s) [noise=true , address=104.36.82.5:4857 , piece_id=7FM7H6OEK7ULAK472NM4P3VQ5UHV6VHV7BOIIDWFGMLG4C4F6JGQ, seg_pos={0 0}, upload=211204]: Done uploading piece. err: ecclient: upload failed (node:1qjxNyfMA6ZjUurQZty2L8U5SUGo6wcLfCb3KVbmWn3ZaDXzQz, address:104.36.82.5:4857): protocol: rpc: dial tcp 104.36.82.5:4857: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:34.234848552 (202h55m56.193945798s) [noise=true , address=104.36.82.60:4352 , piece_id=CSG6QHVS3X4ZYNPLB7XTJVNHLOO25743ATYLF2PIIF22PNSFEEWQ, seg_pos={0 0}, upload=211204]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: rpc: dial tcp 104.36.82.60:4352: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:34.234855720 (202h55m56.193952969s) [noise=true , address=104.36.82.34:4563 , piece_id=O6XS6ZW4TBSFXYJSFK6IZR3J5KPHFWJGV5YVSEKAI3NYLA5XAXRA, seg_pos={0 0}, upload=211204]: Done uploading piece. err: ecclient: upload failed (node:1cDeVufqmqEpR5hjFrhgFmsS7TYeTnhEx3Xyet9ck39sj7Eeiz, address:104.36.82.34:4563): protocol: rpc: dial tcp 104.36.82.34:4563: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:34.234916909 (202h55m56.194014156s) [noise=true , address=104.36.82.18:4384 , piece_id=ACUJDCRZ7D6RAAVPE4G7PUPD7KBASLAYNJ6JNXZFFBKVLOYRJRBA, seg_pos={0 0}, upload=211204]: Done uploading piece. err: ecclient: upload failed (node:12hwbJ9oH6qJD6rucYWqk6BZStAtpM6GY98kC94Tb1edJbpxARE, address:104.36.82.18:4384): protocol: rpc: dial tcp 104.36.82.18:4384: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:34.234971668 (202h55m56.194068906s) [noise=true , address=104.36.82.35:4609 , piece_id=25IQQMRZH7LUC2PBORIR3TEE3C74CLKEZGTAAB2C3RI2375YKKKQ, seg_pos={0 0}, upload=211204]: Done uploading piece. err: ecclient: upload failed (node:12amujuu3Q56TUCgLoyfNLK2KYvkdhWAXJSCJMD1h5hNbNsunbP, address:104.36.82.35:4609): protocol: rpc: dial tcp 104.36.82.35:4609: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:34.234989146 (202h55m56.194086393s) [noise=true , address=104.36.82.58:4286 , piece_id=JWFVQA4FLL3CG6VWEI6EHPBWDWBI7KJ5REVVSJ23HHCTH6HILTSA, seg_pos={0 0}, upload=211204]: Done uploading piece. err: ecclient: upload failed (node:12v2vJnE92VrJXQrbxnAxANHCVASuXiBKMUZECMVmEWEjsZRtsQ, address:104.36.82.58:4286): protocol: write tcp 139.59.144.22:33342->104.36.82.58:4286: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:34.235016097 (202h55m56.194113335s) [noise=true , address=104.36.82.3:4790 , piece_id=2QXJSMA4FMHVR2LBE4QRYMKRWO6H6MESP3W6KCAGGYFCD7LOZKRQ, seg_pos={0 0}, upload=211204]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:104.36.82.3:4790): protocol: rpc: dial tcp 104.36.82.3:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:34.235035425 (202h55m56.194132681s) [seg_pos={0 0}, upload=211204]: Done waiting for segment. successful: 7 optimal: 14 errs: <nil> Aug 17 04:21:34.333629252 (202h55m56.292726491s) [upload=211204]: Done uploading segments... Aug 17 04:21:34.333660145 (202h55m56.292757383s) [upload=211204]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 7) Duration: 1m0.10228965s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.001286933s FAIL (2025-08-17 04:21:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001315714s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.109744607s FAIL (2025-08-17 04:21:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:20:51.225055801 (202h55m13.184153037s) [upload=211207]: Starting upload Aug 17 04:20:51.225124072 (202h55m13.184221327s) [upload=211207]: Done starting upload Aug 17 04:20:51.225502463 (202h55m13.184599699s) [upload=211207]: Uploading segments... Aug 17 04:20:51.225572551 (202h55m13.184669797s) [upload=211207]: Got next segment. Inline: false Aug 17 04:20:51.225603749 (202h55m13.184701002s) [upload=211207]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:20:51.225609099 (202h55m13.184706345s) [upload=211207]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:20:51.482816084 (202h55m13.44191334s) [seg_pos={0 0}, upload=211207]: Begin upload segment... Aug 17 04:20:51.482884139 (202h55m13.441981392s) [seg_pos={0 0}, upload=211207]: Done begin upload segment. Aug 17 04:20:51.483014816 (202h55m13.442112079s) [noise=true , address=104.36.82.16:4295 , piece_id=SVLFFTXFDI6OJPW2MPKTL6W4B5K7RMJTWENL7NVW7X4PVX66JDTA, seg_pos={0 0}, upload=211207]: Uploading piece... Aug 17 04:20:51.483158953 (202h55m13.44225621s) [noise=true , address=104.36.82.8:4969 , piece_id=OLHBBQRVYK73PSVTXJFZG3CE4TVO4IFU4HTI4ZZK5E7RUVT3UCVQ, seg_pos={0 0}, upload=211207]: Uploading piece... Aug 17 04:20:51.483561070 (202h55m13.442658316s) [noise=true , address=45.91.116.4:4937 , piece_id=756INEZRD5QLI655P3EQUQV632L5DL424I3Q26B64MBBHNM7YNFA, seg_pos={0 0}, upload=211207]: Uploading piece... Aug 17 04:20:51.484561167 (202h55m13.443658417s) [noise=true , address=77.90.31.14:4663 , piece_id=Z5MLWFIMCYSVA74AGWQ3YVKKYATZHQXMRNVYO2LVG262SGAMUJOQ, seg_pos={0 0}, upload=211207]: Uploading piece... Aug 17 04:20:51.485081301 (202h55m13.444178548s) [noise=true , address=104.36.82.3:4767 , piece_id=LFAJCIUZH4X4MSHQG25KFFEC4G3WVNF3FCGKL75QTPJYWVCET2BA, seg_pos={0 0}, upload=211207]: Uploading piece... Aug 17 04:20:51.485527265 (202h55m13.44462452s) [noise=true , address=77.90.31.11:4554 , piece_id=PLN2S3TDCDCVGKDDI2RYMR2ELRJAQMWVMNIQAQPXHME2JXKZDVVQ, seg_pos={0 0}, upload=211207]: Uploading piece... Aug 17 04:20:51.486112685 (202h55m13.44520994s) [noise=true , address=45.91.116.7:4009 , piece_id=DBNY4CRGEYK6TBOMP76TEWMEG7RCAFB6SIFIFTF7DDBIMFY5AXTA, seg_pos={0 0}, upload=211207]: Uploading piece... Aug 17 04:20:51.486647432 (202h55m13.445744681s) [noise=true , address=45.91.116.12:4225 , piece_id=6J6Z77HFTRJ5SEI3ZKXFUYOJ5B3PQSKCEVGARBZYGIBOTW5JMRQA, seg_pos={0 0}, upload=211207]: Uploading piece... Aug 17 04:20:51.487139829 (202h55m13.446237074s) [noise=true , address=104.36.82.8:4971 , piece_id=XS2O26IIHJXOFURJOVS4QQ4NWTHSKPV2JRHHPJQSDNOXQTQWMUVQ, seg_pos={0 0}, upload=211207]: Uploading piece... Aug 17 04:20:51.487724388 (202h55m13.44682164s) [noise=true , address=45.91.116.4:4938 , piece_id=L7UUCOVNEKGPNKC65GOEYYW3RO23E2HG7X2HDXIOFTMGYDCQKKFA, seg_pos={0 0}, upload=211207]: Uploading piece... Aug 17 04:20:51.488162089 (202h55m13.447259341s) [noise=true , address=104.36.82.2:4731 , piece_id=E7TEUDDPVFIPU7PGGXKOBYSPRTVOUA4GH3EWEDEG22KFAJH25OUA, seg_pos={0 0}, upload=211207]: Uploading piece... Aug 17 04:20:51.488721918 (202h55m13.447819171s) [noise=true , address=77.90.31.12:4592 , piece_id=RRDD5U6LYIHDTLYV2PFTFD5TYXDY5ZODLZIJTZ6HHUKZAEKMSV5A, seg_pos={0 0}, upload=211207]: Uploading piece... Aug 17 04:20:51.489315488 (202h55m13.448412739s) [noise=true , address=45.91.116.11:4117 , piece_id=KUJ5JXLBNXHFYPU7TSZUMLTLINLOUINQPSSZ3REBLBKX5UQEIYUA, seg_pos={0 0}, upload=211207]: Uploading piece... Aug 17 04:20:51.489798231 (202h55m13.448895462s) [noise=true , address=104.36.82.7:4910 , piece_id=ZT4Q7SAVDLZD4U7UEHUVECRPK4BQIX62NYQ55VUCU4OOAEYPHHBQ, seg_pos={0 0}, upload=211207]: Uploading piece... Aug 17 04:20:51.491833053 (202h55m13.4509303s) [noise=true , address=45.91.116.5:4298 , piece_id=NBYYFK4NJ6BNCHR24P6L2ISGOJSU65F33NCIRK6CRL64LANI74WA, seg_pos={0 0}, upload=211207]: Uploading piece... Aug 17 04:20:51.493845021 (202h55m13.452942273s) [noise=true , address=45.91.116.8:4048 , piece_id=P5FNWWMJUUYINZ6XB32NZFN7XRHSIC6LR26CLPLYKN7JHLTD6O6A, seg_pos={0 0}, upload=211207]: Uploading piece... Aug 17 04:20:52.091379494 (202h55m14.050476741s) [upload=211207]: Next returned nil segment Aug 17 04:20:52.091396982 (202h55m14.050494218s) [upload=211207]: Waiting for error group managing segments... Aug 17 04:20:52.223709062 (202h55m14.182806301s) [noise=true , address=45.91.116.7:4009 , piece_id=DBNY4CRGEYK6TBOMP76TEWMEG7RCAFB6SIFIFTF7DDBIMFY5AXTA, seg_pos={0 0}, upload=211207]: Done uploading piece. err: <nil> Aug 17 04:20:52.228352036 (202h55m14.18744927s) [noise=true , address=77.90.31.14:4663 , piece_id=Z5MLWFIMCYSVA74AGWQ3YVKKYATZHQXMRNVYO2LVG262SGAMUJOQ, seg_pos={0 0}, upload=211207]: Done uploading piece. err: <nil> Aug 17 04:20:52.229795065 (202h55m14.188892303s) [noise=true , address=77.90.31.11:4554 , piece_id=PLN2S3TDCDCVGKDDI2RYMR2ELRJAQMWVMNIQAQPXHME2JXKZDVVQ, seg_pos={0 0}, upload=211207]: Done uploading piece. err: <nil> Aug 17 04:20:52.233197378 (202h55m14.192294616s) [noise=true , address=45.91.116.4:4938 , piece_id=L7UUCOVNEKGPNKC65GOEYYW3RO23E2HG7X2HDXIOFTMGYDCQKKFA, seg_pos={0 0}, upload=211207]: Done uploading piece. err: <nil> Aug 17 04:20:52.234746782 (202h55m14.193844016s) [noise=true , address=45.91.116.4:4937 , piece_id=756INEZRD5QLI655P3EQUQV632L5DL424I3Q26B64MBBHNM7YNFA, seg_pos={0 0}, upload=211207]: Done uploading piece. err: <nil> Aug 17 04:20:52.235122707 (202h55m14.194219954s) [noise=true , address=45.91.116.11:4117 , piece_id=KUJ5JXLBNXHFYPU7TSZUMLTLINLOUINQPSSZ3REBLBKX5UQEIYUA, seg_pos={0 0}, upload=211207]: Done uploading piece. err: <nil> Aug 17 04:20:52.237236021 (202h55m14.196333257s) [noise=true , address=45.91.116.12:4225 , piece_id=6J6Z77HFTRJ5SEI3ZKXFUYOJ5B3PQSKCEVGARBZYGIBOTW5JMRQA, seg_pos={0 0}, upload=211207]: Done uploading piece. err: <nil> Aug 17 04:20:52.239376003 (202h55m14.198473234s) [noise=true , address=45.91.116.8:4048 , piece_id=P5FNWWMJUUYINZ6XB32NZFN7XRHSIC6LR26CLPLYKN7JHLTD6O6A, seg_pos={0 0}, upload=211207]: Done uploading piece. err: <nil> Aug 17 04:20:52.250358888 (202h55m14.209456126s) [noise=true , address=45.91.116.5:4298 , piece_id=NBYYFK4NJ6BNCHR24P6L2ISGOJSU65F33NCIRK6CRL64LANI74WA, seg_pos={0 0}, upload=211207]: Done uploading piece. err: <nil> Aug 17 04:20:52.323560116 (202h55m14.282657355s) [noise=true , address=77.90.31.12:4592 , piece_id=RRDD5U6LYIHDTLYV2PFTFD5TYXDY5ZODLZIJTZ6HHUKZAEKMSV5A, seg_pos={0 0}, upload=211207]: Done uploading piece. err: <nil> Aug 17 04:21:20.955961343 (202h55m42.915058589s) [noise=true , address=104.36.82.8:4971 , piece_id=XS2O26IIHJXOFURJOVS4QQ4NWTHSKPV2JRHHPJQSDNOXQTQWMUVQ, seg_pos={0 0}, upload=211207]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:55270->104.36.82.8:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:20.955976989 (202h55m42.915074227s) [noise=true , address=104.36.82.2:4731 , piece_id=E7TEUDDPVFIPU7PGGXKOBYSPRTVOUA4GH3EWEDEG22KFAJH25OUA, seg_pos={0 0}, upload=211207]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:104.36.82.2:4731): protocol: write tcp 139.59.144.22:53180->104.36.82.2:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:20.956032274 (202h55m42.91512951s) [noise=true , address=104.36.82.7:4910 , piece_id=ZT4Q7SAVDLZD4U7UEHUVECRPK4BQIX62NYQ55VUCU4OOAEYPHHBQ, seg_pos={0 0}, upload=211207]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: write tcp 139.59.144.22:51938->104.36.82.7:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:20.956223867 (202h55m42.915321112s) [noise=true , address=104.36.82.8:4969 , piece_id=OLHBBQRVYK73PSVTXJFZG3CE4TVO4IFU4HTI4ZZK5E7RUVT3UCVQ, seg_pos={0 0}, upload=211207]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 139.59.144.22:55832->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:20.956283102 (202h55m42.915380348s) [noise=true , address=104.36.82.3:4767 , piece_id=LFAJCIUZH4X4MSHQG25KFFEC4G3WVNF3FCGKL75QTPJYWVCET2BA, seg_pos={0 0}, upload=211207]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:40944->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:20.956289108 (202h55m42.915386353s) [noise=true , address=104.36.82.16:4295 , piece_id=SVLFFTXFDI6OJPW2MPKTL6W4B5K7RMJTWENL7NVW7X4PVX66JDTA, seg_pos={0 0}, upload=211207]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 139.59.144.22:37436->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:20.956374514 (202h55m42.915471745s) [seg_pos={0 0}, upload=211207]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:21:21.063230771 (202h55m43.022328018s) [upload=211207]: Done uploading segments... Aug 17 04:21:21.063280152 (202h55m43.022377397s) [upload=211207]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.109771539s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.000849631s FAIL (2025-08-17 04:21:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000875469s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X....................................................X] duration: 1m0.106131747s FAIL (2025-08-17 04:20:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 04:19:46.652111018 (202h54m8.611208266s) [upload=211193]: Starting upload Aug 17 04:19:46.652179501 (202h54m8.611276759s) [upload=211193]: Done starting upload Aug 17 04:19:46.652513318 (202h54m8.611610563s) [upload=211193]: Uploading segments... Aug 17 04:19:46.652736385 (202h54m8.61183363s) [upload=211193]: Got next segment. Inline: false Aug 17 04:19:46.652818929 (202h54m8.611916164s) [upload=211193]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:19:46.652839539 (202h54m8.611936784s) [upload=211193]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:19:46.856264332 (202h54m8.815361568s) [seg_pos={0 0}, upload=211193]: Begin upload segment... Aug 17 04:19:46.856339233 (202h54m8.815436495s) [seg_pos={0 0}, upload=211193]: Done begin upload segment. Aug 17 04:19:46.856366702 (202h54m8.815463965s) [noise=true , address=104.36.82.4:4836 , piece_id=I53EPJSKJW4QIE43JRJYGFLNMGRWTZACY5Y3ZERLI2H4D4PBXQUQ, seg_pos={0 0}, upload=211193]: Uploading piece... Aug 17 04:19:46.856412507 (202h54m8.81550976s) [noise=true , address=64.49.13.169:4706 , piece_id=W5LSYNGNFGM6MSYAE7KKEMNKX4ZOSVU2SN6HNBHZ2YNJSNJLFNSQ, seg_pos={0 0}, upload=211193]: Uploading piece... Aug 17 04:19:46.856869990 (202h54m8.815967235s) [noise=true , address=79.127.201.214:4320 , piece_id=UMR3JLGZ466L4WURWIQ2OQOP3B2PNUVBAHTTELHFWLFSLFFMP75A, seg_pos={0 0}, upload=211193]: Uploading piece... Aug 17 04:19:46.856889884 (202h54m8.815987137s) [noise=true , address=45.91.116.13:4184 , piece_id=IMO3H4UPZMUBMID3XMTI3AIDN2HJSNGP7CHG2B4BGZMV37N2YNBA, seg_pos={0 0}, upload=211193]: Uploading piece... Aug 17 04:19:46.857180738 (202h54m8.816277969s) [noise=true , address=104.36.82.19:4409 , piece_id=ZP5XX5C3IUTODWJS6X3JRZSNCXG3KN5RBYA76JO3NVVWTP6ZKW3Q, seg_pos={0 0}, upload=211193]: Uploading piece... Aug 17 04:19:46.857212154 (202h54m8.816309402s) [noise=true , address=79.127.205.234:4595 , piece_id=F7PM4OWCQZACTPGXZE6LZUCZ37URMEG5WDJZR5DXTNPPX2W7TR7A, seg_pos={0 0}, upload=211193]: Uploading piece... Aug 17 04:19:46.857609912 (202h54m8.816707166s) [noise=true , address=104.36.82.60:4344 , piece_id=XS5N5N6VIGRVNSK4QXRRRPMLG6EJF3HNYOXMS3EZEDTHODP2YOUA, seg_pos={0 0}, upload=211193]: Uploading piece... Aug 17 04:19:46.858166327 (202h54m8.817263576s) [noise=true , address=104.36.82.44:4893 , piece_id=AVRRIZOX7FANZZUS6RKK5DSOPA3VPRH3YEOESMYQAX2S43O4MPFQ, seg_pos={0 0}, upload=211193]: Uploading piece... Aug 17 04:19:46.858505383 (202h54m8.817602623s) [noise=true , address=79.127.201.215:4356 , piece_id=NGGF2ZG4FMTE4EJ6QDHUYSWAHESPQCV4A34QNHIM33KDFW2LLYRQ, seg_pos={0 0}, upload=211193]: Uploading piece... Aug 17 04:19:46.858616960 (202h54m8.817714212s) [noise=true , address=104.36.82.21:4506 , piece_id=2QDKSQ3IWSRQFROKVHDYPYYDQEO3UFSSY4T7OJIMO6LGEHXTT5XA, seg_pos={0 0}, upload=211193]: Uploading piece... Aug 17 04:19:46.858944703 (202h54m8.818041948s) [noise=true , address=79.127.219.48:4093 , piece_id=UGBVP6Q4YQHO7OSZJN3Z6SMHSMH5RVNVGLZ5SG5PPOC4VSGXYSGQ, seg_pos={0 0}, upload=211193]: Uploading piece... Aug 17 04:19:46.859013326 (202h54m8.818110581s) [noise=true , address=77.90.31.18:4840 , piece_id=TUDRN5BLOJDS26DFYO6VZGK5J3XAQED3SNN5DFBWSAUOC5F6IXAA, seg_pos={0 0}, upload=211193]: Uploading piece... Aug 17 04:19:46.859245539 (202h54m8.818342786s) [noise=true , address=156.146.43.225:4468 , piece_id=XOTDVHSSZTFETTT5Q2WGCYR35IWLD4QBE67WQOMFE7V323PHNSZQ, seg_pos={0 0}, upload=211193]: Uploading piece... Aug 17 04:19:46.859331420 (202h54m8.818428659s) [noise=true , address=104.36.82.18:4389 , piece_id=JP5BOHJUJW7XYCVNTRDORWHL5BKXZDTPO43LCTRJXCIJBFLQWN4A, seg_pos={0 0}, upload=211193]: Uploading piece... Aug 17 04:19:46.859879515 (202h54m8.81897682s) [noise=true , address=77.90.31.21:4904 , piece_id=HJXU5SFP4KPQU6PA5STX4Q7KHRKPU7HROXHT6IWTAI2TEG6PWF2Q, seg_pos={0 0}, upload=211193]: Uploading piece... Aug 17 04:19:46.860601342 (202h54m8.819698581s) [noise=true , address=104.36.82.5:4843 , piece_id=F4UUYS5QWD5CUDRCXRGUWDFZ22237QNJUKS2B2XRGZ7XBSNFS5HQ, seg_pos={0 0}, upload=211193]: Uploading piece... Aug 17 04:19:46.861212208 (202h54m8.820309457s) [noise=true , address=104.36.82.23:4476 , piece_id=OGEXIBGQDOUOPJFCXN5CQXICUPRZU3DPDA3DZXMBRUH6YF6KGPXQ, seg_pos={0 0}, upload=211193]: Uploading piece... Aug 17 04:19:46.861588851 (202h54m8.820686099s) [noise=true , address=66.23.193.175:4562 , piece_id=RLXYDUSPXXRYAKVKNW67G2MMBA6XQR3LSM7DIUBYE7HA53EIRBHQ, seg_pos={0 0}, upload=211193]: Uploading piece... Aug 17 04:19:47.449877655 (202h54m9.4089749s) [upload=211193]: Next returned nil segment Aug 17 04:19:47.449896815 (202h54m9.40899406s) [upload=211193]: Waiting for error group managing segments... Aug 17 04:19:47.612078354 (202h54m9.5711756s) [noise=true , address=45.91.116.13:4184 , piece_id=IMO3H4UPZMUBMID3XMTI3AIDN2HJSNGP7CHG2B4BGZMV37N2YNBA, seg_pos={0 0}, upload=211193]: Done uploading piece. err: <nil> Aug 17 04:19:47.663658826 (202h54m9.622756071s) [noise=true , address=79.127.219.48:4093 , piece_id=UGBVP6Q4YQHO7OSZJN3Z6SMHSMH5RVNVGLZ5SG5PPOC4VSGXYSGQ, seg_pos={0 0}, upload=211193]: Done uploading piece. err: <nil> Aug 17 04:19:47.693543126 (202h54m9.652640358s) [noise=true , address=77.90.31.18:4840 , piece_id=TUDRN5BLOJDS26DFYO6VZGK5J3XAQED3SNN5DFBWSAUOC5F6IXAA, seg_pos={0 0}, upload=211193]: Done uploading piece. err: <nil> Aug 17 04:19:47.728905930 (202h54m9.688003169s) [noise=true , address=77.90.31.21:4904 , piece_id=HJXU5SFP4KPQU6PA5STX4Q7KHRKPU7HROXHT6IWTAI2TEG6PWF2Q, seg_pos={0 0}, upload=211193]: Done uploading piece. err: <nil> Aug 17 04:19:47.837466899 (202h54m9.796564131s) [noise=true , address=79.127.201.214:4320 , piece_id=UMR3JLGZ466L4WURWIQ2OQOP3B2PNUVBAHTTELHFWLFSLFFMP75A, seg_pos={0 0}, upload=211193]: Done uploading piece. err: <nil> Aug 17 04:19:47.848557753 (202h54m9.807654999s) [noise=true , address=79.127.201.215:4356 , piece_id=NGGF2ZG4FMTE4EJ6QDHUYSWAHESPQCV4A34QNHIM33KDFW2LLYRQ, seg_pos={0 0}, upload=211193]: Done uploading piece. err: <nil> Aug 17 04:19:47.870095833 (202h54m9.829193082s) [noise=true , address=156.146.43.225:4468 , piece_id=XOTDVHSSZTFETTT5Q2WGCYR35IWLD4QBE67WQOMFE7V323PHNSZQ, seg_pos={0 0}, upload=211193]: Done uploading piece. err: <nil> Aug 17 04:19:48.100106612 (202h54m10.059203851s) [noise=true , address=66.23.193.175:4562 , piece_id=RLXYDUSPXXRYAKVKNW67G2MMBA6XQR3LSM7DIUBYE7HA53EIRBHQ, seg_pos={0 0}, upload=211193]: Done uploading piece. err: <nil> Aug 17 04:19:48.201215505 (202h54m10.160312743s) [noise=true , address=79.127.205.234:4595 , piece_id=F7PM4OWCQZACTPGXZE6LZUCZ37URMEG5WDJZR5DXTNPPX2W7TR7A, seg_pos={0 0}, upload=211193]: Done uploading piece. err: <nil> Aug 17 04:19:49.277263382 (202h54m11.23636063s) [noise=true , address=64.49.13.169:4706 , piece_id=W5LSYNGNFGM6MSYAE7KKEMNKX4ZOSVU2SN6HNBHZ2YNJSNJLFNSQ, seg_pos={0 0}, upload=211193]: Done uploading piece. err: <nil> Aug 17 04:20:46.374532495 (202h55m8.333629753s) [noise=true , address=104.36.82.4:4836 , piece_id=I53EPJSKJW4QIE43JRJYGFLNMGRWTZACY5Y3ZERLI2H4D4PBXQUQ, seg_pos={0 0}, upload=211193]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:104.36.82.4:4836): protocol: rpc: dial tcp 104.36.82.4:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:46.374620661 (202h55m8.333717912s) [noise=true , address=104.36.82.5:4843 , piece_id=F4UUYS5QWD5CUDRCXRGUWDFZ22237QNJUKS2B2XRGZ7XBSNFS5HQ, seg_pos={0 0}, upload=211193]: Done uploading piece. err: ecclient: upload failed (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS, address:104.36.82.5:4843): protocol: rpc: dial tcp 104.36.82.5:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:46.374681468 (202h55m8.333778719s) [noise=true , address=104.36.82.23:4476 , piece_id=OGEXIBGQDOUOPJFCXN5CQXICUPRZU3DPDA3DZXMBRUH6YF6KGPXQ, seg_pos={0 0}, upload=211193]: Done uploading piece. err: ecclient: upload failed (node:1foWwyGzFenUicfY7cTU5dmdDJsvXGR3YXv1rf3QRXMUco59TV, address:104.36.82.23:4476): protocol: rpc: dial tcp 104.36.82.23:4476: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:46.374756589 (202h55m8.333853836s) [noise=true , address=104.36.82.19:4409 , piece_id=ZP5XX5C3IUTODWJS6X3JRZSNCXG3KN5RBYA76JO3NVVWTP6ZKW3Q, seg_pos={0 0}, upload=211193]: Done uploading piece. err: ecclient: upload failed (node:125CLa5WEhb2C5y9CJtyy38JfqhUaWusA4Vr4FPdkXMFoMQ7SiP, address:104.36.82.19:4409): protocol: rpc: dial tcp 104.36.82.19:4409: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:46.374812575 (202h55m8.333909821s) [noise=true , address=104.36.82.60:4344 , piece_id=XS5N5N6VIGRVNSK4QXRRRPMLG6EJF3HNYOXMS3EZEDTHODP2YOUA, seg_pos={0 0}, upload=211193]: Done uploading piece. err: ecclient: upload failed (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK, address:104.36.82.60:4344): protocol: rpc: dial tcp 104.36.82.60:4344: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:46.374846583 (202h55m8.33394383s) [noise=true , address=104.36.82.44:4893 , piece_id=AVRRIZOX7FANZZUS6RKK5DSOPA3VPRH3YEOESMYQAX2S43O4MPFQ, seg_pos={0 0}, upload=211193]: Done uploading piece. err: ecclient: upload failed (node:17VMFZ2T8jrSJVtchciJ9SnyrR6JstvcMoXrN5Q3zTD1aQU1oJ, address:104.36.82.44:4893): protocol: rpc: dial tcp 104.36.82.44:4893: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:46.374859113 (202h55m8.333956354s) [noise=true , address=104.36.82.18:4389 , piece_id=JP5BOHJUJW7XYCVNTRDORWHL5BKXZDTPO43LCTRJXCIJBFLQWN4A, seg_pos={0 0}, upload=211193]: Done uploading piece. err: ecclient: upload failed (node:1mMqCXPNzFx4cnu8Nb3gNKcJFJ4tGJjnkwY2EVPjUBnwNVcM9R, address:104.36.82.18:4389): protocol: rpc: dial tcp 104.36.82.18:4389: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:46.374930009 (202h55m8.334027255s) [noise=true , address=104.36.82.21:4506 , piece_id=2QDKSQ3IWSRQFROKVHDYPYYDQEO3UFSSY4T7OJIMO6LGEHXTT5XA, seg_pos={0 0}, upload=211193]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: write tcp 139.59.144.22:41426->104.36.82.21:4506: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:46.374966463 (202h55m8.334063718s) [seg_pos={0 0}, upload=211193]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:20:46.478643135 (202h55m8.437740385s) [upload=211193]: Done uploading segments... Aug 17 04:20:46.478677930 (202h55m8.43777517s) [upload=211193]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.106168254s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.002009887s FAIL (2025-08-17 04:20:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.002084741s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.106716376s FAIL (2025-08-17 04:20:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== RN3FZVIA, seg_pos={0 0}, upload=211187]: Uploading piece... Aug 17 04:19:34.445189364 (202h53m56.404286619s) [noise=true , address=104.36.82.5:4859 , piece_id=GEZ3SRFDDDJBITRZEHRSBE3YF22XN6DB6WDZFY7NVOSW5E4JBOQQ, seg_pos={0 0}, upload=211187]: Uploading piece... Aug 17 04:19:34.445577497 (202h53m56.404674731s) [noise=true , address=104.36.82.60:4367 , piece_id=IIDEC5VEVNNL7PEXX3FFHO2PQUHIJZNP3FB7Z7OK23C3PJ5CKGXA, seg_pos={0 0}, upload=211187]: Uploading piece... Aug 17 04:19:34.445751761 (202h53m56.40484902s) [noise=true , address=45.91.116.11:4127 , piece_id=S7U5C464FFU32FWIERTGEXIHH67QB264EHH2JHQZELQGNRTFV2YA, seg_pos={0 0}, upload=211187]: Uploading piece... Aug 17 04:19:34.445845024 (202h53m56.404942269s) [noise=true , address=104.36.82.3:4795 , piece_id=XHUE7DCHQUD3LHGQB7QHMZOU6IKL2H7MNWAQK7U7Q44BBCSTR7PQ, seg_pos={0 0}, upload=211187]: Uploading piece... Aug 17 04:19:34.446264901 (202h53m56.405362167s) [noise=true , address=104.36.82.19:4436 , piece_id=Z2C4GB7NEQEJHQ2C2GVGKPUYXLDG42CNPA5XPGFOTGQYJ2LXFMJQ, seg_pos={0 0}, upload=211187]: Uploading piece... Aug 17 04:19:34.446364484 (202h53m56.405461739s) [noise=true , address=104.36.82.59:4323 , piece_id=WPSZGDFL25SKSLNNHXWMFA3TMFEPJEMVT5CQLJEQ5WCBM5CIY3LA, seg_pos={0 0}, upload=211187]: Uploading piece... Aug 17 04:19:34.446637412 (202h53m56.405734663s) [noise=true , address=77.90.31.16:4741 , piece_id=KL5QG5VX3ZDO326OUAYQQHKSXJA66K3LQN2ZIYLJ3P37PHGKJJTA, seg_pos={0 0}, upload=211187]: Uploading piece... Aug 17 04:19:34.446845163 (202h53m56.405942417s) [noise=true , address=45.91.116.9:4275 , piece_id=IOKOC5BFKVU5MXCT2GRLXYA7RKYSX2YOXTLLRL3AXONWRYQG3AWQ, seg_pos={0 0}, upload=211187]: Uploading piece... Aug 17 04:19:34.447076259 (202h53m56.406173515s) [noise=true , address=77.90.31.19:4850 , piece_id=ARNZ6GEPYOMHEBNJU5WILA4UMJIBGWAPRDCPJFHLKU5VERPSDHDA, seg_pos={0 0}, upload=211187]: Uploading piece... Aug 17 04:19:34.447348029 (202h53m56.406445286s) [noise=true , address=45.91.116.6:4994 , piece_id=JR3SRBDPTAJFT426PGULZ6HOR7R7DYLAI3RL2W2LSTAXQACPGS6Q, seg_pos={0 0}, upload=211187]: Uploading piece... Aug 17 04:19:34.447421856 (202h53m56.40651911s) [noise=true , address=79.127.201.215:4351 , piece_id=E2GV7LXJKS3DWFL4NC6EZL5SYPBG3MPNXTTYBSRY7SH7PQ6DEQNA, seg_pos={0 0}, upload=211187]: Uploading piece... Aug 17 04:19:34.447701734 (202h53m56.40679898s) [noise=true , address=45.91.116.7:4041 , piece_id=AACTZ44VDVVLDFRUMHQEMF77ZCMRYOHBRY2GAV24KP3ZTEJSX2QQ, seg_pos={0 0}, upload=211187]: Uploading piece... Aug 17 04:19:34.447798528 (202h53m56.406895765s) [noise=true , address=104.36.82.8:4004 , piece_id=JXCI73FEB7VZPD34XIQXGK4GHXCV7C2RVI7FYRFXVRDCSLZK7CFQ, seg_pos={0 0}, upload=211187]: Uploading piece... Aug 17 04:19:34.448275456 (202h53m56.40737271s) [noise=true , address=104.36.82.23:4453 , piece_id=JJIOGOFDV2ASFBRJZBUMKEHEJXYU3DBBL7GWIRUJVTBHPO4XFBMQ, seg_pos={0 0}, upload=211187]: Uploading piece... Aug 17 04:19:34.448712848 (202h53m56.407810105s) [noise=true , address=77.90.31.11:4558 , piece_id=R5KHQUK3LOJ5CYU4X2KRXM4GLLFRFNT5W7BVEZY54EBFOMVZVRWQ, seg_pos={0 0}, upload=211187]: Uploading piece... Aug 17 04:19:34.448970137 (202h53m56.408067382s) [noise=true , address=104.36.82.44:4902 , piece_id=KDRGGSXKPAMLAIXLLAXH5SXPHVKWHNNGX3KXP3ENNO5PJJEZN2FA, seg_pos={0 0}, upload=211187]: Uploading piece... Aug 17 04:19:34.449212594 (202h53m56.408309842s) [noise=true , address=77.90.31.21:4894 , piece_id=SDIJJLUZGCO4ZK6M5PT4PJ5RQITFHJT7WXULKNSRZ2TSTOKYUFZQ, seg_pos={0 0}, upload=211187]: Uploading piece... Aug 17 04:19:35.001893073 (202h53m56.96099034s) [upload=211187]: Next returned nil segment Aug 17 04:19:35.001983330 (202h53m56.961080578s) [upload=211187]: Waiting for error group managing segments... Aug 17 04:19:35.147525304 (202h53m57.106622541s) [noise=true , address=45.91.116.11:4127 , piece_id=S7U5C464FFU32FWIERTGEXIHH67QB264EHH2JHQZELQGNRTFV2YA, seg_pos={0 0}, upload=211187]: Done uploading piece. err: <nil> Aug 17 04:19:35.147969407 (202h53m57.107066645s) [noise=true , address=45.91.116.6:4994 , piece_id=JR3SRBDPTAJFT426PGULZ6HOR7R7DYLAI3RL2W2LSTAXQACPGS6Q, seg_pos={0 0}, upload=211187]: Done uploading piece. err: <nil> Aug 17 04:19:35.152020970 (202h53m57.111118206s) [noise=true , address=45.91.116.9:4275 , piece_id=IOKOC5BFKVU5MXCT2GRLXYA7RKYSX2YOXTLLRL3AXONWRYQG3AWQ, seg_pos={0 0}, upload=211187]: Done uploading piece. err: <nil> Aug 17 04:19:35.163839973 (202h53m57.122937212s) [noise=true , address=77.90.31.11:4558 , piece_id=R5KHQUK3LOJ5CYU4X2KRXM4GLLFRFNT5W7BVEZY54EBFOMVZVRWQ, seg_pos={0 0}, upload=211187]: Done uploading piece. err: <nil> Aug 17 04:19:35.168915038 (202h53m57.128012275s) [noise=true , address=45.91.116.7:4041 , piece_id=AACTZ44VDVVLDFRUMHQEMF77ZCMRYOHBRY2GAV24KP3ZTEJSX2QQ, seg_pos={0 0}, upload=211187]: Done uploading piece. err: <nil> Aug 17 04:19:35.179620750 (202h53m57.138717987s) [noise=true , address=77.90.31.16:4741 , piece_id=KL5QG5VX3ZDO326OUAYQQHKSXJA66K3LQN2ZIYLJ3P37PHGKJJTA, seg_pos={0 0}, upload=211187]: Done uploading piece. err: <nil> Aug 17 04:19:35.273532418 (202h53m57.232629661s) [noise=true , address=77.90.31.19:4850 , piece_id=ARNZ6GEPYOMHEBNJU5WILA4UMJIBGWAPRDCPJFHLKU5VERPSDHDA, seg_pos={0 0}, upload=211187]: Done uploading piece. err: <nil> Aug 17 04:19:35.317426529 (202h53m57.276523766s) [noise=true , address=77.90.31.21:4894 , piece_id=SDIJJLUZGCO4ZK6M5PT4PJ5RQITFHJT7WXULKNSRZ2TSTOKYUFZQ, seg_pos={0 0}, upload=211187]: Done uploading piece. err: <nil> Aug 17 04:19:35.703708366 (202h53m57.662805612s) [noise=true , address=66.23.193.175:4564 , piece_id=PVWVRDYBVBRMO4RDESGBMKQCEGKGVO2L6Z6ZYKWQN7GTRN3FZVIA, seg_pos={0 0}, upload=211187]: Done uploading piece. err: <nil> Aug 17 04:19:36.431424912 (202h53m58.390522153s) [noise=true , address=79.127.201.215:4351 , piece_id=E2GV7LXJKS3DWFL4NC6EZL5SYPBG3MPNXTTYBSRY7SH7PQ6DEQNA, seg_pos={0 0}, upload=211187]: Done uploading piece. err: <nil> Aug 17 04:20:33.975903424 (202h54m55.935000669s) [noise=true , address=104.36.82.60:4367 , piece_id=IIDEC5VEVNNL7PEXX3FFHO2PQUHIJZNP3FB7Z7OK23C3PJ5CKGXA, seg_pos={0 0}, upload=211187]: Done uploading piece. err: ecclient: upload failed (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK, address:104.36.82.60:4367): protocol: rpc: dial tcp 104.36.82.60:4367: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:33.975954341 (202h54m55.935051572s) [noise=true , address=104.36.82.59:4323 , piece_id=WPSZGDFL25SKSLNNHXWMFA3TMFEPJEMVT5CQLJEQ5WCBM5CIY3LA, seg_pos={0 0}, upload=211187]: Done uploading piece. err: ecclient: upload failed (node:12HLr6Ha1ysWYbUJ8He7RofMFCz8qVuJMBFkfJgnTUVw1kmXckz, address:104.36.82.59:4323): protocol: rpc: dial tcp 104.36.82.59:4323: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:33.975989338 (202h54m55.935086584s) [noise=true , address=104.36.82.23:4453 , piece_id=JJIOGOFDV2ASFBRJZBUMKEHEJXYU3DBBL7GWIRUJVTBHPO4XFBMQ, seg_pos={0 0}, upload=211187]: Done uploading piece. err: ecclient: upload failed (node:1ktniPbRibXzsTSNW9MbWWxymt9gZKU8sDgTjixc6nYzT9c5tJ, address:104.36.82.23:4453): protocol: rpc: dial tcp 104.36.82.23:4453: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:33.976029775 (202h54m55.93512702s) [noise=true , address=104.36.82.44:4902 , piece_id=KDRGGSXKPAMLAIXLLAXH5SXPHVKWHNNGX3KXP3ENNO5PJJEZN2FA, seg_pos={0 0}, upload=211187]: Done uploading piece. err: ecclient: upload failed (node:1Be9T5kXUwLU3C6wBeKFV3Q7aZFbZDTNG1oaEft8V6rmySUcEE, address:104.36.82.44:4902): protocol: write tcp 139.59.144.22:48654->104.36.82.44:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:33.976042192 (202h54m55.935139423s) [noise=true , address=104.36.82.8:4004 , piece_id=JXCI73FEB7VZPD34XIQXGK4GHXCV7C2RVI7FYRFXVRDCSLZK7CFQ, seg_pos={0 0}, upload=211187]: Done uploading piece. err: ecclient: upload failed (node:12BdVmmJ1L1sEMcHXTAdXBXU6dmdhUE9VT74jCNdZj5bMqJWxqp, address:104.36.82.8:4004): protocol: rpc: dial tcp 104.36.82.8:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:33.976072516 (202h54m55.935169765s) [noise=true , address=104.36.82.5:4859 , piece_id=GEZ3SRFDDDJBITRZEHRSBE3YF22XN6DB6WDZFY7NVOSW5E4JBOQQ, seg_pos={0 0}, upload=211187]: Done uploading piece. err: ecclient: upload failed (node:1EiYA6gHiLyHsMfPevj2gLqmqttu1knLw3AM8SbWWtgKEf7Csu, address:104.36.82.5:4859): protocol: rpc: dial tcp 104.36.82.5:4859: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:33.976124763 (202h54m55.935222009s) [noise=true , address=104.36.82.3:4795 , piece_id=XHUE7DCHQUD3LHGQB7QHMZOU6IKL2H7MNWAQK7U7Q44BBCSTR7PQ, seg_pos={0 0}, upload=211187]: Done uploading piece. err: ecclient: upload failed (node:1rL9dLWui62A3vRgBSNw7QQf7g2uubwW85to4ioCrhyao2Pupo, address:104.36.82.3:4795): protocol: rpc: dial tcp 104.36.82.3:4795: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:33.976132445 (202h54m55.935229677s) [noise=true , address=104.36.82.19:4436 , piece_id=Z2C4GB7NEQEJHQ2C2GVGKPUYXLDG42CNPA5XPGFOTGQYJ2LXFMJQ, seg_pos={0 0}, upload=211187]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: write tcp 139.59.144.22:38904->104.36.82.19:4436: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:33.976174658 (202h54m55.935271911s) [seg_pos={0 0}, upload=211187]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:20:34.079798097 (202h54m56.038895344s) [upload=211187]: Done uploading segments... Aug 17 04:20:34.079843671 (202h54m56.03894092s) [upload=211187]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.106742955s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.389620537s FAIL (2025-08-17 04:18:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.389654978s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.4595956s FAIL (2025-08-17 04:18:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.459624784s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.930971391s FAIL (2025-08-17 04:17:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.930991791s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 2.645734108s FAIL (2025-08-17 04:17:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.646319445s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.789178124s FAIL (2025-08-17 04:17:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.789232498s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.476859291s FAIL (2025-08-17 04:16:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.476894285s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 2.502726188s FAIL (2025-08-17 04:16:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.502747544s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 2.754796s FAIL (2025-08-17 04:15:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.754841034s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.835231715s FAIL (2025-08-17 04:15:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.835268694s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.43710733s FAIL (2025-08-17 04:15:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.437214078s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 3.042675336s FAIL (2025-08-17 04:15:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.042734858s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.386210712s FAIL (2025-08-17 04:13:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.386260613s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 2.772255539s FAIL (2025-08-17 04:13:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.772285104s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.486205762s FAIL (2025-08-17 04:13:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.486234602s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration: 3.101795107s FAIL (2025-08-17 04:12:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.101822597s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.908660417s FAIL (2025-08-17 04:12:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.908702437s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......XX....................................................] duration: 2.657550057s FAIL (2025-08-17 04:12:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.657590106s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.343448262s FAIL (2025-08-17 04:11:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.343495666s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.662539889s FAIL (2025-08-17 04:11:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.662606429s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......XX...................................................] duration: 2.796290714s FAIL (2025-08-17 04:11:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.79633837s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)