Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................................XXXX...........] duration: 2.336214649s FAIL (2025-08-17 04:32:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Aug 17 04:22:36.193175039 (202h56m58.152272297s) [upload=211241]: Starting upload Aug 17 04:22:36.193282733 (202h56m58.152379987s) [upload=211241]: Done starting upload Aug 17 04:22:36.193533603 (202h56m58.152630849s) [upload=211241]: Uploading segments... Aug 17 04:22:36.193824638 (202h56m58.152921893s) [upload=211241]: Got next segment. Inline: false Aug 17 04:22:36.193891646 (202h56m58.152988897s) [upload=211241]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:22:36.193898487 (202h56m58.15299574s) [upload=211241]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:22:36.387313731 (202h56m58.346410984s) [seg_pos={0 0}, upload=211241]: Begin upload segment... Aug 17 04:22:36.387391508 (202h56m58.346488754s) [seg_pos={0 0}, upload=211241]: Done begin upload segment. Aug 17 04:22:36.387453494 (202h56m58.34655085s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.387549035 (202h56m58.346646289s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388292176 (202h56m58.347389421s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388564570 (202h56m58.347661824s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388775483 (202h56m58.347872728s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388976887 (202h56m58.348074132s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389048807 (202h56m58.348146055s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389269495 (202h56m58.348366743s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389303339 (202h56m58.348400583s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389577693 (202h56m58.348674943s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389582267 (202h56m58.348679515s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389868029 (202h56m58.348965273s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389899606 (202h56m58.348996853s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390181444 (202h56m58.349278692s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390456412 (202h56m58.349553657s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.396850594 (202h56m58.355947842s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:37.003486038 (202h56m58.962583287s) [upload=211241]: Next returned nil segment Aug 17 04:22:37.003509888 (202h56m58.962607148s) [upload=211241]: Waiting for error group managing segments... Aug 17 04:22:37.141087302 (202h56m59.100184557s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.147724589 (202h56m59.106821827s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148191395 (202h56m59.107288641s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148249179 (202h56m59.107346437s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.154197920 (202h56m59.113295159s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155088755 (202h56m59.11418599s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155737855 (202h56m59.114835102s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.158542805 (202h56m59.117640044s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.163837059 (202h56m59.122934309s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.225669823 (202h56m59.184767059s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.249261694 (202h56m59.20835894s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:23:05.920185881 (202h57m27.879283118s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: rpc: dial tcp 104.36.82.10:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920277695 (202h57m27.879374941s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: rpc: dial tcp 104.36.82.15:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920476811 (202h57m27.87957406s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: rpc: dial tcp 104.36.82.7:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920550680 (202h57m27.879647915s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920687798 (202h57m27.879785037s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:52446->104.36.82.8:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 04:23:05.920736972 (202h57m27.879834217s) [seg_pos={0 0}, upload=211241]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Aug 17 04:23:06.024578477 (202h57m27.983675715s) [upload=211241]: Done uploading segments... Aug 17 04:23:06.024615107 (202h57m27.983712354s) [upload=211241]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.336255185s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................XXX................] duration: 2.553708228s FAIL (2025-08-17 04:32:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 17 04:21:58.824378996 (202h56m20.783476241s) [upload=211227]: Starting upload Aug 17 04:21:58.824453728 (202h56m20.783550973s) [upload=211227]: Done starting upload Aug 17 04:21:58.824533712 (202h56m20.783630943s) [upload=211227]: Uploading segments... Aug 17 04:21:58.824814622 (202h56m20.783911863s) [upload=211227]: Got next segment. Inline: false Aug 17 04:21:58.824870724 (202h56m20.78396797s) [upload=211227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:21:58.824876461 (202h56m20.783973706s) [upload=211227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:21:59.025160521 (202h56m20.984257797s) [seg_pos={0 0}, upload=211227]: Begin upload segment... Aug 17 04:21:59.025238019 (202h56m20.984335266s) [seg_pos={0 0}, upload=211227]: Done begin upload segment. Aug 17 04:21:59.025273686 (202h56m20.98437092s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025331026 (202h56m20.984428309s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025756251 (202h56m20.984853496s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025907425 (202h56m20.98500468s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026127117 (202h56m20.98522437s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026350346 (202h56m20.985447594s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026436670 (202h56m20.985533916s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026714369 (202h56m20.985811614s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026788896 (202h56m20.985886144s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027068637 (202h56m20.986165882s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027120600 (202h56m20.986217846s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027420110 (202h56m20.986517355s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027425372 (202h56m20.98652262s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027839122 (202h56m20.986936376s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.028247142 (202h56m20.987344396s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030005732 (202h56m20.989102986s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030432900 (202h56m20.989530147s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.031846491 (202h56m20.990943746s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.588920323 (202h56m21.548017569s) [upload=211227]: Next returned nil segment Aug 17 04:21:59.588944741 (202h56m21.548041972s) [upload=211227]: Waiting for error group managing segments... Aug 17 04:21:59.741262552 (202h56m21.700359791s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.742768290 (202h56m21.701865528s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.777622259 (202h56m21.736719503s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.808692811 (202h56m21.767790057s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.831688332 (202h56m21.790785579s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.871745680 (202h56m21.830842918s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.883314367 (202h56m21.84241162s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.890311438 (202h56m21.849408688s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.940441979 (202h56m21.899539228s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.976656973 (202h56m21.935754218s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.283185203 (202h56m22.242282448s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.284822985 (202h56m22.243920238s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.446988508 (202h56m22.406085753s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:58.545488378 (202h57m20.504585625s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr, address:104.36.82.5:4866): protocol: write tcp 139.59.144.22:57798->104.36.82.5:4866: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545674721 (202h57m20.504771972s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 139.59.144.22:57190->104.36.82.23:4447: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545824338 (202h57m20.504921585s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1, address:104.36.82.44:4901): protocol: write tcp 139.59.144.22:54098->104.36.82.44:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545853626 (202h57m20.504950858s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12ASAeTMBu4sY1GuZiSuwTXmhPksA1Te2BVZdcoE9QvpLAmcQX3, address:104.36.82.35:4603): protocol: write tcp 139.59.144.22:49164->104.36.82.35:4603: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545907146 (202h57m20.505004393s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: write tcp 139.59.144.22:43990->104.36.82.60:4365: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545989950 (202h57m20.505087195s) [seg_pos={0 0}, upload=211227]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 04:22:58.646357515 (202h57m20.605454767s) [upload=211227]: Done uploading segments... Aug 17 04:22:58.646391150 (202h57m20.605488398s) [upload=211227]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.553742562s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [............................................XX..............] duration: 2.971263535s FAIL (2025-08-17 04:31:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.971365942s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................................XXXX..........] duration: 2.359787595s FAIL (2025-08-17 04:31:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Aug 17 04:22:36.193175039 (202h56m58.152272297s) [upload=211241]: Starting upload Aug 17 04:22:36.193282733 (202h56m58.152379987s) [upload=211241]: Done starting upload Aug 17 04:22:36.193533603 (202h56m58.152630849s) [upload=211241]: Uploading segments... Aug 17 04:22:36.193824638 (202h56m58.152921893s) [upload=211241]: Got next segment. Inline: false Aug 17 04:22:36.193891646 (202h56m58.152988897s) [upload=211241]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:22:36.193898487 (202h56m58.15299574s) [upload=211241]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:22:36.387313731 (202h56m58.346410984s) [seg_pos={0 0}, upload=211241]: Begin upload segment... Aug 17 04:22:36.387391508 (202h56m58.346488754s) [seg_pos={0 0}, upload=211241]: Done begin upload segment. Aug 17 04:22:36.387453494 (202h56m58.34655085s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.387549035 (202h56m58.346646289s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388292176 (202h56m58.347389421s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388564570 (202h56m58.347661824s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388775483 (202h56m58.347872728s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388976887 (202h56m58.348074132s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389048807 (202h56m58.348146055s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389269495 (202h56m58.348366743s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389303339 (202h56m58.348400583s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389577693 (202h56m58.348674943s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389582267 (202h56m58.348679515s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389868029 (202h56m58.348965273s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389899606 (202h56m58.348996853s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390181444 (202h56m58.349278692s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390456412 (202h56m58.349553657s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.396850594 (202h56m58.355947842s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:37.003486038 (202h56m58.962583287s) [upload=211241]: Next returned nil segment Aug 17 04:22:37.003509888 (202h56m58.962607148s) [upload=211241]: Waiting for error group managing segments... Aug 17 04:22:37.141087302 (202h56m59.100184557s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.147724589 (202h56m59.106821827s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148191395 (202h56m59.107288641s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148249179 (202h56m59.107346437s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.154197920 (202h56m59.113295159s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155088755 (202h56m59.11418599s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155737855 (202h56m59.114835102s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.158542805 (202h56m59.117640044s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.163837059 (202h56m59.122934309s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.225669823 (202h56m59.184767059s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.249261694 (202h56m59.20835894s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:23:05.920185881 (202h57m27.879283118s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: rpc: dial tcp 104.36.82.10:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920277695 (202h57m27.879374941s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: rpc: dial tcp 104.36.82.15:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920476811 (202h57m27.87957406s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: rpc: dial tcp 104.36.82.7:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920550680 (202h57m27.879647915s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920687798 (202h57m27.879785037s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:52446->104.36.82.8:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 04:23:05.920736972 (202h57m27.879834217s) [seg_pos={0 0}, upload=211241]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Aug 17 04:23:06.024578477 (202h57m27.983675715s) [upload=211241]: Done uploading segments... Aug 17 04:23:06.024615107 (202h57m27.983712354s) [upload=211241]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.359887965s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..............................................XXXXX.........] duration: 2.971496148s FAIL (2025-08-17 04:31:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.971524114s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................................XXXX............] duration: 2.420713345s FAIL (2025-08-17 04:31:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277217340 (202h56m58.236314588s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277444018 (202h56m58.236541264s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277693944 (202h56m58.236791191s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277957287 (202h56m58.237054531s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278192843 (202h56m58.237290088s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278363507 (202h56m58.237460782s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278887870 (202h56m58.237985121s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279265735 (202h56m58.238362991s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279639846 (202h56m58.238737091s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279971414 (202h56m58.239068662s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280041108 (202h56m58.239138356s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280352792 (202h56m58.239450046s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280683314 (202h56m58.239780567s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280969649 (202h56m58.240066896s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281241193 (202h56m58.240338438s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281508639 (202h56m58.240605887s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.837376257 (202h56m58.796473489s) [upload=211240]: Next returned nil segment Aug 17 04:22:36.837399552 (202h56m58.796496798s) [upload=211240]: Waiting for error group managing segments... Aug 17 04:22:36.989047654 (202h56m58.948144887s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991378317 (202h56m58.950475552s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991998552 (202h56m58.951095785s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.010955158 (202h56m58.970052395s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.017235751 (202h56m58.97633299s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.045617264 (202h56m59.004714496s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.080535376 (202h56m59.039632621s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.121242355 (202h56m59.080339591s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.296208684 (202h56m59.255305934s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.528205336 (202h56m59.487302581s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:23:35.761418960 (202h57m57.720516213s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1okrDyM4TfzYN3p8zUjTqbw2661rKQQqZdP1ptAKzzTQ82C7cN, address:104.36.82.60:4341): protocol: rpc: dial tcp 104.36.82.60:4341: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761509741 (202h57m57.720606987s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1i62GRFASf7UPVGhUofmRkz3zZSyhq4fDycgyYtfD7pHR3w91v, address:104.36.82.34:4556): protocol: write tcp 139.59.144.22:37150->104.36.82.34:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761574924 (202h57m57.720672174s) [noise=true , address=104.36.82.17:4333 , piece_id=CGHZEP2ZKWES3L6XH4LSFLOFKYLQPYJZ56G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1xRdTZoEk3NsnS445GcdEvwAdDzNLwUYSfgjB7bBSGRx9L9mrU, address:104.36.82.17:4333): protocol: write tcp 139.59.144.22:47980->104.36.82.17:4333: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761620359 (202h57m57.720717605s) [noise=true , address=104.36.82.58:4293 , piece_id=IY3EUM3SZ42OZA42BC2FSVTLTHU5POSMOOQT6MTLHNPZ6FEYE2EQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:15i783VxxzbriD7LZCbFWNBY8uCKUffXtmWE2eu2VupgE1t9bZ, address:104.36.82.58:4293): protocol: write tcp 139.59.144.22:39172->104.36.82.58:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761661063 (202h57m57.720758308s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:12aETKgTzKw8fyYqbeDA2JzfxZnWn12xUPYK1eRt6TvBPPdRJDp, address:104.36.82.36:4630): protocol: write tcp 139.59.144.22:51970->104.36.82.36:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761681516 (202h57m57.720778766s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 139.59.144.22:35702->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761728458 (202h57m57.720825693s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:48846->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761769868 (202h57m57.720867115s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1hbt8HKTunpPKyAzH4XBwGQqTKzcgqPpoWwKZm9JMtaK8P7aVp, address:104.36.82.8:4992): protocol: write tcp 139.59.144.22:40976->104.36.82.8:4992: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761795616 (202h57m57.720892851s) [seg_pos={0 0}, upload=211240]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:35.867721911 (202h57m57.826819144s) [upload=211240]: Done uploading segments... Aug 17 04:23:35.867749737 (202h57m57.826846969s) [upload=211240]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 2.420765099s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................................XXX...............] duration: 2.593516711s FAIL (2025-08-17 04:31:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 17 04:21:58.824378996 (202h56m20.783476241s) [upload=211227]: Starting upload Aug 17 04:21:58.824453728 (202h56m20.783550973s) [upload=211227]: Done starting upload Aug 17 04:21:58.824533712 (202h56m20.783630943s) [upload=211227]: Uploading segments... Aug 17 04:21:58.824814622 (202h56m20.783911863s) [upload=211227]: Got next segment. Inline: false Aug 17 04:21:58.824870724 (202h56m20.78396797s) [upload=211227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:21:58.824876461 (202h56m20.783973706s) [upload=211227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:21:59.025160521 (202h56m20.984257797s) [seg_pos={0 0}, upload=211227]: Begin upload segment... Aug 17 04:21:59.025238019 (202h56m20.984335266s) [seg_pos={0 0}, upload=211227]: Done begin upload segment. Aug 17 04:21:59.025273686 (202h56m20.98437092s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025331026 (202h56m20.984428309s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025756251 (202h56m20.984853496s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025907425 (202h56m20.98500468s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026127117 (202h56m20.98522437s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026350346 (202h56m20.985447594s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026436670 (202h56m20.985533916s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026714369 (202h56m20.985811614s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026788896 (202h56m20.985886144s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027068637 (202h56m20.986165882s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027120600 (202h56m20.986217846s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027420110 (202h56m20.986517355s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027425372 (202h56m20.98652262s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027839122 (202h56m20.986936376s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.028247142 (202h56m20.987344396s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030005732 (202h56m20.989102986s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030432900 (202h56m20.989530147s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.031846491 (202h56m20.990943746s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.588920323 (202h56m21.548017569s) [upload=211227]: Next returned nil segment Aug 17 04:21:59.588944741 (202h56m21.548041972s) [upload=211227]: Waiting for error group managing segments... Aug 17 04:21:59.741262552 (202h56m21.700359791s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.742768290 (202h56m21.701865528s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.777622259 (202h56m21.736719503s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.808692811 (202h56m21.767790057s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.831688332 (202h56m21.790785579s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.871745680 (202h56m21.830842918s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.883314367 (202h56m21.84241162s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.890311438 (202h56m21.849408688s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.940441979 (202h56m21.899539228s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.976656973 (202h56m21.935754218s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.283185203 (202h56m22.242282448s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.284822985 (202h56m22.243920238s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.446988508 (202h56m22.406085753s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:58.545488378 (202h57m20.504585625s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr, address:104.36.82.5:4866): protocol: write tcp 139.59.144.22:57798->104.36.82.5:4866: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545674721 (202h57m20.504771972s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 139.59.144.22:57190->104.36.82.23:4447: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545824338 (202h57m20.504921585s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1, address:104.36.82.44:4901): protocol: write tcp 139.59.144.22:54098->104.36.82.44:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545853626 (202h57m20.504950858s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12ASAeTMBu4sY1GuZiSuwTXmhPksA1Te2BVZdcoE9QvpLAmcQX3, address:104.36.82.35:4603): protocol: write tcp 139.59.144.22:49164->104.36.82.35:4603: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545907146 (202h57m20.505004393s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: write tcp 139.59.144.22:43990->104.36.82.60:4365: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545989950 (202h57m20.505087195s) [seg_pos={0 0}, upload=211227]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 04:22:58.646357515 (202h57m20.605454767s) [upload=211227]: Done uploading segments... Aug 17 04:22:58.646391150 (202h57m20.605488398s) [upload=211227]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.593634339s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................................XXXX.........] duration: 2.411682219s FAIL (2025-08-17 04:30:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Aug 17 04:22:36.193175039 (202h56m58.152272297s) [upload=211241]: Starting upload Aug 17 04:22:36.193282733 (202h56m58.152379987s) [upload=211241]: Done starting upload Aug 17 04:22:36.193533603 (202h56m58.152630849s) [upload=211241]: Uploading segments... Aug 17 04:22:36.193824638 (202h56m58.152921893s) [upload=211241]: Got next segment. Inline: false Aug 17 04:22:36.193891646 (202h56m58.152988897s) [upload=211241]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:22:36.193898487 (202h56m58.15299574s) [upload=211241]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:22:36.387313731 (202h56m58.346410984s) [seg_pos={0 0}, upload=211241]: Begin upload segment... Aug 17 04:22:36.387391508 (202h56m58.346488754s) [seg_pos={0 0}, upload=211241]: Done begin upload segment. Aug 17 04:22:36.387453494 (202h56m58.34655085s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.387549035 (202h56m58.346646289s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388292176 (202h56m58.347389421s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388564570 (202h56m58.347661824s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388775483 (202h56m58.347872728s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388976887 (202h56m58.348074132s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389048807 (202h56m58.348146055s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389269495 (202h56m58.348366743s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389303339 (202h56m58.348400583s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389577693 (202h56m58.348674943s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389582267 (202h56m58.348679515s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389868029 (202h56m58.348965273s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389899606 (202h56m58.348996853s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390181444 (202h56m58.349278692s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390456412 (202h56m58.349553657s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.396850594 (202h56m58.355947842s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:37.003486038 (202h56m58.962583287s) [upload=211241]: Next returned nil segment Aug 17 04:22:37.003509888 (202h56m58.962607148s) [upload=211241]: Waiting for error group managing segments... Aug 17 04:22:37.141087302 (202h56m59.100184557s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.147724589 (202h56m59.106821827s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148191395 (202h56m59.107288641s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148249179 (202h56m59.107346437s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.154197920 (202h56m59.113295159s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155088755 (202h56m59.11418599s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155737855 (202h56m59.114835102s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.158542805 (202h56m59.117640044s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.163837059 (202h56m59.122934309s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.225669823 (202h56m59.184767059s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.249261694 (202h56m59.20835894s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:23:05.920185881 (202h57m27.879283118s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: rpc: dial tcp 104.36.82.10:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920277695 (202h57m27.879374941s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: rpc: dial tcp 104.36.82.15:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920476811 (202h57m27.87957406s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: rpc: dial tcp 104.36.82.7:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920550680 (202h57m27.879647915s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920687798 (202h57m27.879785037s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:52446->104.36.82.8:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 04:23:05.920736972 (202h57m27.879834217s) [seg_pos={0 0}, upload=211241]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Aug 17 04:23:06.024578477 (202h57m27.983675715s) [upload=211241]: Done uploading segments... Aug 17 04:23:06.024615107 (202h57m27.983712354s) [upload=211241]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.411738488s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.............................................XX.............] duration: 3.057345208s FAIL (2025-08-17 04:30:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.057373924s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...............................................XXXXX........] duration: 2.932933419s FAIL (2025-08-17 04:30:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.933021262s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................................XXXX...........] duration: 2.79409109s FAIL (2025-08-17 04:30:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277217340 (202h56m58.236314588s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277444018 (202h56m58.236541264s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277693944 (202h56m58.236791191s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277957287 (202h56m58.237054531s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278192843 (202h56m58.237290088s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278363507 (202h56m58.237460782s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278887870 (202h56m58.237985121s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279265735 (202h56m58.238362991s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279639846 (202h56m58.238737091s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279971414 (202h56m58.239068662s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280041108 (202h56m58.239138356s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280352792 (202h56m58.239450046s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280683314 (202h56m58.239780567s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280969649 (202h56m58.240066896s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281241193 (202h56m58.240338438s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281508639 (202h56m58.240605887s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.837376257 (202h56m58.796473489s) [upload=211240]: Next returned nil segment Aug 17 04:22:36.837399552 (202h56m58.796496798s) [upload=211240]: Waiting for error group managing segments... Aug 17 04:22:36.989047654 (202h56m58.948144887s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991378317 (202h56m58.950475552s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991998552 (202h56m58.951095785s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.010955158 (202h56m58.970052395s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.017235751 (202h56m58.97633299s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.045617264 (202h56m59.004714496s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.080535376 (202h56m59.039632621s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.121242355 (202h56m59.080339591s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.296208684 (202h56m59.255305934s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.528205336 (202h56m59.487302581s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:23:35.761418960 (202h57m57.720516213s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1okrDyM4TfzYN3p8zUjTqbw2661rKQQqZdP1ptAKzzTQ82C7cN, address:104.36.82.60:4341): protocol: rpc: dial tcp 104.36.82.60:4341: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761509741 (202h57m57.720606987s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1i62GRFASf7UPVGhUofmRkz3zZSyhq4fDycgyYtfD7pHR3w91v, address:104.36.82.34:4556): protocol: write tcp 139.59.144.22:37150->104.36.82.34:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761574924 (202h57m57.720672174s) [noise=true , address=104.36.82.17:4333 , piece_id=CGHZEP2ZKWES3L6XH4LSFLOFKYLQPYJZ56G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1xRdTZoEk3NsnS445GcdEvwAdDzNLwUYSfgjB7bBSGRx9L9mrU, address:104.36.82.17:4333): protocol: write tcp 139.59.144.22:47980->104.36.82.17:4333: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761620359 (202h57m57.720717605s) [noise=true , address=104.36.82.58:4293 , piece_id=IY3EUM3SZ42OZA42BC2FSVTLTHU5POSMOOQT6MTLHNPZ6FEYE2EQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:15i783VxxzbriD7LZCbFWNBY8uCKUffXtmWE2eu2VupgE1t9bZ, address:104.36.82.58:4293): protocol: write tcp 139.59.144.22:39172->104.36.82.58:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761661063 (202h57m57.720758308s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:12aETKgTzKw8fyYqbeDA2JzfxZnWn12xUPYK1eRt6TvBPPdRJDp, address:104.36.82.36:4630): protocol: write tcp 139.59.144.22:51970->104.36.82.36:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761681516 (202h57m57.720778766s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 139.59.144.22:35702->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761728458 (202h57m57.720825693s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:48846->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761769868 (202h57m57.720867115s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1hbt8HKTunpPKyAzH4XBwGQqTKzcgqPpoWwKZm9JMtaK8P7aVp, address:104.36.82.8:4992): protocol: write tcp 139.59.144.22:40976->104.36.82.8:4992: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761795616 (202h57m57.720892851s) [seg_pos={0 0}, upload=211240]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:35.867721911 (202h57m57.826819144s) [upload=211240]: Done uploading segments... Aug 17 04:23:35.867749737 (202h57m57.826846969s) [upload=211240]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 2.79416493s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................................XXXX........] duration: 2.429812453s FAIL (2025-08-17 04:30:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Aug 17 04:22:36.193175039 (202h56m58.152272297s) [upload=211241]: Starting upload Aug 17 04:22:36.193282733 (202h56m58.152379987s) [upload=211241]: Done starting upload Aug 17 04:22:36.193533603 (202h56m58.152630849s) [upload=211241]: Uploading segments... Aug 17 04:22:36.193824638 (202h56m58.152921893s) [upload=211241]: Got next segment. Inline: false Aug 17 04:22:36.193891646 (202h56m58.152988897s) [upload=211241]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:22:36.193898487 (202h56m58.15299574s) [upload=211241]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:22:36.387313731 (202h56m58.346410984s) [seg_pos={0 0}, upload=211241]: Begin upload segment... Aug 17 04:22:36.387391508 (202h56m58.346488754s) [seg_pos={0 0}, upload=211241]: Done begin upload segment. Aug 17 04:22:36.387453494 (202h56m58.34655085s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.387549035 (202h56m58.346646289s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388292176 (202h56m58.347389421s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388564570 (202h56m58.347661824s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388775483 (202h56m58.347872728s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388976887 (202h56m58.348074132s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389048807 (202h56m58.348146055s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389269495 (202h56m58.348366743s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389303339 (202h56m58.348400583s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389577693 (202h56m58.348674943s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389582267 (202h56m58.348679515s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389868029 (202h56m58.348965273s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389899606 (202h56m58.348996853s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390181444 (202h56m58.349278692s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390456412 (202h56m58.349553657s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.396850594 (202h56m58.355947842s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:37.003486038 (202h56m58.962583287s) [upload=211241]: Next returned nil segment Aug 17 04:22:37.003509888 (202h56m58.962607148s) [upload=211241]: Waiting for error group managing segments... Aug 17 04:22:37.141087302 (202h56m59.100184557s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.147724589 (202h56m59.106821827s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148191395 (202h56m59.107288641s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148249179 (202h56m59.107346437s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.154197920 (202h56m59.113295159s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155088755 (202h56m59.11418599s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155737855 (202h56m59.114835102s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.158542805 (202h56m59.117640044s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.163837059 (202h56m59.122934309s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.225669823 (202h56m59.184767059s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.249261694 (202h56m59.20835894s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:23:05.920185881 (202h57m27.879283118s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: rpc: dial tcp 104.36.82.10:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920277695 (202h57m27.879374941s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: rpc: dial tcp 104.36.82.15:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920476811 (202h57m27.87957406s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: rpc: dial tcp 104.36.82.7:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920550680 (202h57m27.879647915s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920687798 (202h57m27.879785037s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:52446->104.36.82.8:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 04:23:05.920736972 (202h57m27.879834217s) [seg_pos={0 0}, upload=211241]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Aug 17 04:23:06.024578477 (202h57m27.983675715s) [upload=211241]: Done uploading segments... Aug 17 04:23:06.024615107 (202h57m27.983712354s) [upload=211241]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.429860069s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..............................................XX............] duration: 3.332382342s FAIL (2025-08-17 04:29:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.332401724s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................................XXX..............] duration: 2.451735981s FAIL (2025-08-17 04:29:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 17 04:21:58.824378996 (202h56m20.783476241s) [upload=211227]: Starting upload Aug 17 04:21:58.824453728 (202h56m20.783550973s) [upload=211227]: Done starting upload Aug 17 04:21:58.824533712 (202h56m20.783630943s) [upload=211227]: Uploading segments... Aug 17 04:21:58.824814622 (202h56m20.783911863s) [upload=211227]: Got next segment. Inline: false Aug 17 04:21:58.824870724 (202h56m20.78396797s) [upload=211227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:21:58.824876461 (202h56m20.783973706s) [upload=211227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:21:59.025160521 (202h56m20.984257797s) [seg_pos={0 0}, upload=211227]: Begin upload segment... Aug 17 04:21:59.025238019 (202h56m20.984335266s) [seg_pos={0 0}, upload=211227]: Done begin upload segment. Aug 17 04:21:59.025273686 (202h56m20.98437092s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025331026 (202h56m20.984428309s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025756251 (202h56m20.984853496s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025907425 (202h56m20.98500468s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026127117 (202h56m20.98522437s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026350346 (202h56m20.985447594s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026436670 (202h56m20.985533916s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026714369 (202h56m20.985811614s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026788896 (202h56m20.985886144s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027068637 (202h56m20.986165882s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027120600 (202h56m20.986217846s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027420110 (202h56m20.986517355s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027425372 (202h56m20.98652262s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027839122 (202h56m20.986936376s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.028247142 (202h56m20.987344396s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030005732 (202h56m20.989102986s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030432900 (202h56m20.989530147s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.031846491 (202h56m20.990943746s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.588920323 (202h56m21.548017569s) [upload=211227]: Next returned nil segment Aug 17 04:21:59.588944741 (202h56m21.548041972s) [upload=211227]: Waiting for error group managing segments... Aug 17 04:21:59.741262552 (202h56m21.700359791s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.742768290 (202h56m21.701865528s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.777622259 (202h56m21.736719503s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.808692811 (202h56m21.767790057s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.831688332 (202h56m21.790785579s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.871745680 (202h56m21.830842918s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.883314367 (202h56m21.84241162s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.890311438 (202h56m21.849408688s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.940441979 (202h56m21.899539228s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.976656973 (202h56m21.935754218s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.283185203 (202h56m22.242282448s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.284822985 (202h56m22.243920238s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.446988508 (202h56m22.406085753s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:58.545488378 (202h57m20.504585625s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr, address:104.36.82.5:4866): protocol: write tcp 139.59.144.22:57798->104.36.82.5:4866: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545674721 (202h57m20.504771972s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 139.59.144.22:57190->104.36.82.23:4447: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545824338 (202h57m20.504921585s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1, address:104.36.82.44:4901): protocol: write tcp 139.59.144.22:54098->104.36.82.44:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545853626 (202h57m20.504950858s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12ASAeTMBu4sY1GuZiSuwTXmhPksA1Te2BVZdcoE9QvpLAmcQX3, address:104.36.82.35:4603): protocol: write tcp 139.59.144.22:49164->104.36.82.35:4603: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545907146 (202h57m20.505004393s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: write tcp 139.59.144.22:43990->104.36.82.60:4365: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545989950 (202h57m20.505087195s) [seg_pos={0 0}, upload=211227]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 04:22:58.646357515 (202h57m20.605454767s) [upload=211227]: Done uploading segments... Aug 17 04:22:58.646391150 (202h57m20.605488398s) [upload=211227]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.451769658s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................................XXXX.......] duration: 2.336661176s FAIL (2025-08-17 04:29:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Aug 17 04:22:36.193175039 (202h56m58.152272297s) [upload=211241]: Starting upload Aug 17 04:22:36.193282733 (202h56m58.152379987s) [upload=211241]: Done starting upload Aug 17 04:22:36.193533603 (202h56m58.152630849s) [upload=211241]: Uploading segments... Aug 17 04:22:36.193824638 (202h56m58.152921893s) [upload=211241]: Got next segment. Inline: false Aug 17 04:22:36.193891646 (202h56m58.152988897s) [upload=211241]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:22:36.193898487 (202h56m58.15299574s) [upload=211241]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:22:36.387313731 (202h56m58.346410984s) [seg_pos={0 0}, upload=211241]: Begin upload segment... Aug 17 04:22:36.387391508 (202h56m58.346488754s) [seg_pos={0 0}, upload=211241]: Done begin upload segment. Aug 17 04:22:36.387453494 (202h56m58.34655085s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.387549035 (202h56m58.346646289s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388292176 (202h56m58.347389421s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388564570 (202h56m58.347661824s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388775483 (202h56m58.347872728s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388976887 (202h56m58.348074132s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389048807 (202h56m58.348146055s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389269495 (202h56m58.348366743s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389303339 (202h56m58.348400583s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389577693 (202h56m58.348674943s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389582267 (202h56m58.348679515s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389868029 (202h56m58.348965273s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389899606 (202h56m58.348996853s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390181444 (202h56m58.349278692s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390456412 (202h56m58.349553657s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.396850594 (202h56m58.355947842s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:37.003486038 (202h56m58.962583287s) [upload=211241]: Next returned nil segment Aug 17 04:22:37.003509888 (202h56m58.962607148s) [upload=211241]: Waiting for error group managing segments... Aug 17 04:22:37.141087302 (202h56m59.100184557s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.147724589 (202h56m59.106821827s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148191395 (202h56m59.107288641s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148249179 (202h56m59.107346437s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.154197920 (202h56m59.113295159s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155088755 (202h56m59.11418599s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155737855 (202h56m59.114835102s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.158542805 (202h56m59.117640044s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.163837059 (202h56m59.122934309s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.225669823 (202h56m59.184767059s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.249261694 (202h56m59.20835894s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:23:05.920185881 (202h57m27.879283118s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: rpc: dial tcp 104.36.82.10:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920277695 (202h57m27.879374941s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: rpc: dial tcp 104.36.82.15:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920476811 (202h57m27.87957406s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: rpc: dial tcp 104.36.82.7:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920550680 (202h57m27.879647915s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920687798 (202h57m27.879785037s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:52446->104.36.82.8:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 04:23:05.920736972 (202h57m27.879834217s) [seg_pos={0 0}, upload=211241]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Aug 17 04:23:06.024578477 (202h57m27.983675715s) [upload=211241]: Done uploading segments... Aug 17 04:23:06.024615107 (202h57m27.983712354s) [upload=211241]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.336693123s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................................XXXX..........] duration: 2.530854544s FAIL (2025-08-17 04:29:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277217340 (202h56m58.236314588s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277444018 (202h56m58.236541264s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277693944 (202h56m58.236791191s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277957287 (202h56m58.237054531s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278192843 (202h56m58.237290088s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278363507 (202h56m58.237460782s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278887870 (202h56m58.237985121s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279265735 (202h56m58.238362991s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279639846 (202h56m58.238737091s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279971414 (202h56m58.239068662s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280041108 (202h56m58.239138356s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280352792 (202h56m58.239450046s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280683314 (202h56m58.239780567s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280969649 (202h56m58.240066896s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281241193 (202h56m58.240338438s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281508639 (202h56m58.240605887s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.837376257 (202h56m58.796473489s) [upload=211240]: Next returned nil segment Aug 17 04:22:36.837399552 (202h56m58.796496798s) [upload=211240]: Waiting for error group managing segments... Aug 17 04:22:36.989047654 (202h56m58.948144887s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991378317 (202h56m58.950475552s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991998552 (202h56m58.951095785s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.010955158 (202h56m58.970052395s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.017235751 (202h56m58.97633299s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.045617264 (202h56m59.004714496s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.080535376 (202h56m59.039632621s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.121242355 (202h56m59.080339591s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.296208684 (202h56m59.255305934s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.528205336 (202h56m59.487302581s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:23:35.761418960 (202h57m57.720516213s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1okrDyM4TfzYN3p8zUjTqbw2661rKQQqZdP1ptAKzzTQ82C7cN, address:104.36.82.60:4341): protocol: rpc: dial tcp 104.36.82.60:4341: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761509741 (202h57m57.720606987s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1i62GRFASf7UPVGhUofmRkz3zZSyhq4fDycgyYtfD7pHR3w91v, address:104.36.82.34:4556): protocol: write tcp 139.59.144.22:37150->104.36.82.34:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761574924 (202h57m57.720672174s) [noise=true , address=104.36.82.17:4333 , piece_id=CGHZEP2ZKWES3L6XH4LSFLOFKYLQPYJZ56G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1xRdTZoEk3NsnS445GcdEvwAdDzNLwUYSfgjB7bBSGRx9L9mrU, address:104.36.82.17:4333): protocol: write tcp 139.59.144.22:47980->104.36.82.17:4333: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761620359 (202h57m57.720717605s) [noise=true , address=104.36.82.58:4293 , piece_id=IY3EUM3SZ42OZA42BC2FSVTLTHU5POSMOOQT6MTLHNPZ6FEYE2EQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:15i783VxxzbriD7LZCbFWNBY8uCKUffXtmWE2eu2VupgE1t9bZ, address:104.36.82.58:4293): protocol: write tcp 139.59.144.22:39172->104.36.82.58:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761661063 (202h57m57.720758308s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:12aETKgTzKw8fyYqbeDA2JzfxZnWn12xUPYK1eRt6TvBPPdRJDp, address:104.36.82.36:4630): protocol: write tcp 139.59.144.22:51970->104.36.82.36:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761681516 (202h57m57.720778766s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 139.59.144.22:35702->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761728458 (202h57m57.720825693s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:48846->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761769868 (202h57m57.720867115s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1hbt8HKTunpPKyAzH4XBwGQqTKzcgqPpoWwKZm9JMtaK8P7aVp, address:104.36.82.8:4992): protocol: write tcp 139.59.144.22:40976->104.36.82.8:4992: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761795616 (202h57m57.720892851s) [seg_pos={0 0}, upload=211240]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:35.867721911 (202h57m57.826819144s) [upload=211240]: Done uploading segments... Aug 17 04:23:35.867749737 (202h57m57.826846969s) [upload=211240]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 2.530884177s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [................................................XXXXX.......] duration: 2.903138081s FAIL (2025-08-17 04:29:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.903228383s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................................XXXX.........] duration: 2.417584278s FAIL (2025-08-17 04:29:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277217340 (202h56m58.236314588s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277444018 (202h56m58.236541264s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277693944 (202h56m58.236791191s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277957287 (202h56m58.237054531s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278192843 (202h56m58.237290088s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278363507 (202h56m58.237460782s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278887870 (202h56m58.237985121s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279265735 (202h56m58.238362991s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279639846 (202h56m58.238737091s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279971414 (202h56m58.239068662s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280041108 (202h56m58.239138356s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280352792 (202h56m58.239450046s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280683314 (202h56m58.239780567s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280969649 (202h56m58.240066896s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281241193 (202h56m58.240338438s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281508639 (202h56m58.240605887s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.837376257 (202h56m58.796473489s) [upload=211240]: Next returned nil segment Aug 17 04:22:36.837399552 (202h56m58.796496798s) [upload=211240]: Waiting for error group managing segments... Aug 17 04:22:36.989047654 (202h56m58.948144887s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991378317 (202h56m58.950475552s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991998552 (202h56m58.951095785s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.010955158 (202h56m58.970052395s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.017235751 (202h56m58.97633299s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.045617264 (202h56m59.004714496s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.080535376 (202h56m59.039632621s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.121242355 (202h56m59.080339591s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.296208684 (202h56m59.255305934s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.528205336 (202h56m59.487302581s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:23:35.761418960 (202h57m57.720516213s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1okrDyM4TfzYN3p8zUjTqbw2661rKQQqZdP1ptAKzzTQ82C7cN, address:104.36.82.60:4341): protocol: rpc: dial tcp 104.36.82.60:4341: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761509741 (202h57m57.720606987s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1i62GRFASf7UPVGhUofmRkz3zZSyhq4fDycgyYtfD7pHR3w91v, address:104.36.82.34:4556): protocol: write tcp 139.59.144.22:37150->104.36.82.34:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761574924 (202h57m57.720672174s) [noise=true , address=104.36.82.17:4333 , piece_id=CGHZEP2ZKWES3L6XH4LSFLOFKYLQPYJZ56G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1xRdTZoEk3NsnS445GcdEvwAdDzNLwUYSfgjB7bBSGRx9L9mrU, address:104.36.82.17:4333): protocol: write tcp 139.59.144.22:47980->104.36.82.17:4333: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761620359 (202h57m57.720717605s) [noise=true , address=104.36.82.58:4293 , piece_id=IY3EUM3SZ42OZA42BC2FSVTLTHU5POSMOOQT6MTLHNPZ6FEYE2EQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:15i783VxxzbriD7LZCbFWNBY8uCKUffXtmWE2eu2VupgE1t9bZ, address:104.36.82.58:4293): protocol: write tcp 139.59.144.22:39172->104.36.82.58:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761661063 (202h57m57.720758308s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:12aETKgTzKw8fyYqbeDA2JzfxZnWn12xUPYK1eRt6TvBPPdRJDp, address:104.36.82.36:4630): protocol: write tcp 139.59.144.22:51970->104.36.82.36:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761681516 (202h57m57.720778766s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 139.59.144.22:35702->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761728458 (202h57m57.720825693s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:48846->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761769868 (202h57m57.720867115s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1hbt8HKTunpPKyAzH4XBwGQqTKzcgqPpoWwKZm9JMtaK8P7aVp, address:104.36.82.8:4992): protocol: write tcp 139.59.144.22:40976->104.36.82.8:4992: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761795616 (202h57m57.720892851s) [seg_pos={0 0}, upload=211240]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:35.867721911 (202h57m57.826819144s) [upload=211240]: Done uploading segments... Aug 17 04:23:35.867749737 (202h57m57.826846969s) [upload=211240]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 2.41762802s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................................XXX.............] duration: 2.533933984s FAIL (2025-08-17 04:29:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 17 04:21:58.824378996 (202h56m20.783476241s) [upload=211227]: Starting upload Aug 17 04:21:58.824453728 (202h56m20.783550973s) [upload=211227]: Done starting upload Aug 17 04:21:58.824533712 (202h56m20.783630943s) [upload=211227]: Uploading segments... Aug 17 04:21:58.824814622 (202h56m20.783911863s) [upload=211227]: Got next segment. Inline: false Aug 17 04:21:58.824870724 (202h56m20.78396797s) [upload=211227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:21:58.824876461 (202h56m20.783973706s) [upload=211227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:21:59.025160521 (202h56m20.984257797s) [seg_pos={0 0}, upload=211227]: Begin upload segment... Aug 17 04:21:59.025238019 (202h56m20.984335266s) [seg_pos={0 0}, upload=211227]: Done begin upload segment. Aug 17 04:21:59.025273686 (202h56m20.98437092s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025331026 (202h56m20.984428309s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025756251 (202h56m20.984853496s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025907425 (202h56m20.98500468s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026127117 (202h56m20.98522437s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026350346 (202h56m20.985447594s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026436670 (202h56m20.985533916s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026714369 (202h56m20.985811614s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026788896 (202h56m20.985886144s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027068637 (202h56m20.986165882s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027120600 (202h56m20.986217846s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027420110 (202h56m20.986517355s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027425372 (202h56m20.98652262s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027839122 (202h56m20.986936376s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.028247142 (202h56m20.987344396s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030005732 (202h56m20.989102986s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030432900 (202h56m20.989530147s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.031846491 (202h56m20.990943746s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.588920323 (202h56m21.548017569s) [upload=211227]: Next returned nil segment Aug 17 04:21:59.588944741 (202h56m21.548041972s) [upload=211227]: Waiting for error group managing segments... Aug 17 04:21:59.741262552 (202h56m21.700359791s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.742768290 (202h56m21.701865528s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.777622259 (202h56m21.736719503s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.808692811 (202h56m21.767790057s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.831688332 (202h56m21.790785579s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.871745680 (202h56m21.830842918s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.883314367 (202h56m21.84241162s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.890311438 (202h56m21.849408688s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.940441979 (202h56m21.899539228s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.976656973 (202h56m21.935754218s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.283185203 (202h56m22.242282448s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.284822985 (202h56m22.243920238s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.446988508 (202h56m22.406085753s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:58.545488378 (202h57m20.504585625s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr, address:104.36.82.5:4866): protocol: write tcp 139.59.144.22:57798->104.36.82.5:4866: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545674721 (202h57m20.504771972s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 139.59.144.22:57190->104.36.82.23:4447: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545824338 (202h57m20.504921585s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1, address:104.36.82.44:4901): protocol: write tcp 139.59.144.22:54098->104.36.82.44:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545853626 (202h57m20.504950858s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12ASAeTMBu4sY1GuZiSuwTXmhPksA1Te2BVZdcoE9QvpLAmcQX3, address:104.36.82.35:4603): protocol: write tcp 139.59.144.22:49164->104.36.82.35:4603: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545907146 (202h57m20.505004393s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: write tcp 139.59.144.22:43990->104.36.82.60:4365: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545989950 (202h57m20.505087195s) [seg_pos={0 0}, upload=211227]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 04:22:58.646357515 (202h57m20.605454767s) [upload=211227]: Done uploading segments... Aug 17 04:22:58.646391150 (202h57m20.605488398s) [upload=211227]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.533968376s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...............................................XX...........] duration: 3.173197545s FAIL (2025-08-17 04:29:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.173218938s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [................................................XX..........] duration: 3.00032492s FAIL (2025-08-17 04:28:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.000390632s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................................XXX............] duration: 2.560607323s FAIL (2025-08-17 04:28:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 17 04:21:58.824378996 (202h56m20.783476241s) [upload=211227]: Starting upload Aug 17 04:21:58.824453728 (202h56m20.783550973s) [upload=211227]: Done starting upload Aug 17 04:21:58.824533712 (202h56m20.783630943s) [upload=211227]: Uploading segments... Aug 17 04:21:58.824814622 (202h56m20.783911863s) [upload=211227]: Got next segment. Inline: false Aug 17 04:21:58.824870724 (202h56m20.78396797s) [upload=211227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:21:58.824876461 (202h56m20.783973706s) [upload=211227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:21:59.025160521 (202h56m20.984257797s) [seg_pos={0 0}, upload=211227]: Begin upload segment... Aug 17 04:21:59.025238019 (202h56m20.984335266s) [seg_pos={0 0}, upload=211227]: Done begin upload segment. Aug 17 04:21:59.025273686 (202h56m20.98437092s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025331026 (202h56m20.984428309s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025756251 (202h56m20.984853496s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025907425 (202h56m20.98500468s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026127117 (202h56m20.98522437s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026350346 (202h56m20.985447594s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026436670 (202h56m20.985533916s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026714369 (202h56m20.985811614s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026788896 (202h56m20.985886144s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027068637 (202h56m20.986165882s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027120600 (202h56m20.986217846s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027420110 (202h56m20.986517355s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027425372 (202h56m20.98652262s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027839122 (202h56m20.986936376s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.028247142 (202h56m20.987344396s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030005732 (202h56m20.989102986s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030432900 (202h56m20.989530147s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.031846491 (202h56m20.990943746s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.588920323 (202h56m21.548017569s) [upload=211227]: Next returned nil segment Aug 17 04:21:59.588944741 (202h56m21.548041972s) [upload=211227]: Waiting for error group managing segments... Aug 17 04:21:59.741262552 (202h56m21.700359791s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.742768290 (202h56m21.701865528s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.777622259 (202h56m21.736719503s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.808692811 (202h56m21.767790057s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.831688332 (202h56m21.790785579s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.871745680 (202h56m21.830842918s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.883314367 (202h56m21.84241162s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.890311438 (202h56m21.849408688s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.940441979 (202h56m21.899539228s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.976656973 (202h56m21.935754218s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.283185203 (202h56m22.242282448s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.284822985 (202h56m22.243920238s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.446988508 (202h56m22.406085753s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:58.545488378 (202h57m20.504585625s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr, address:104.36.82.5:4866): protocol: write tcp 139.59.144.22:57798->104.36.82.5:4866: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545674721 (202h57m20.504771972s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 139.59.144.22:57190->104.36.82.23:4447: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545824338 (202h57m20.504921585s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1, address:104.36.82.44:4901): protocol: write tcp 139.59.144.22:54098->104.36.82.44:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545853626 (202h57m20.504950858s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12ASAeTMBu4sY1GuZiSuwTXmhPksA1Te2BVZdcoE9QvpLAmcQX3, address:104.36.82.35:4603): protocol: write tcp 139.59.144.22:49164->104.36.82.35:4603: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545907146 (202h57m20.505004393s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: write tcp 139.59.144.22:43990->104.36.82.60:4365: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545989950 (202h57m20.505087195s) [seg_pos={0 0}, upload=211227]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 04:22:58.646357515 (202h57m20.605454767s) [upload=211227]: Done uploading segments... Aug 17 04:22:58.646391150 (202h57m20.605488398s) [upload=211227]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.560636506s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.................................................XX.........] duration: 3.02390406s FAIL (2025-08-17 04:28:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.023930413s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.................................................XXXXX......] duration: 2.75876286s FAIL (2025-08-17 04:28:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.758792696s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................................XXXX......] duration: 2.372436997s FAIL (2025-08-17 04:28:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Aug 17 04:22:36.193175039 (202h56m58.152272297s) [upload=211241]: Starting upload Aug 17 04:22:36.193282733 (202h56m58.152379987s) [upload=211241]: Done starting upload Aug 17 04:22:36.193533603 (202h56m58.152630849s) [upload=211241]: Uploading segments... Aug 17 04:22:36.193824638 (202h56m58.152921893s) [upload=211241]: Got next segment. Inline: false Aug 17 04:22:36.193891646 (202h56m58.152988897s) [upload=211241]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:22:36.193898487 (202h56m58.15299574s) [upload=211241]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:22:36.387313731 (202h56m58.346410984s) [seg_pos={0 0}, upload=211241]: Begin upload segment... Aug 17 04:22:36.387391508 (202h56m58.346488754s) [seg_pos={0 0}, upload=211241]: Done begin upload segment. Aug 17 04:22:36.387453494 (202h56m58.34655085s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.387549035 (202h56m58.346646289s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388292176 (202h56m58.347389421s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388564570 (202h56m58.347661824s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388775483 (202h56m58.347872728s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388976887 (202h56m58.348074132s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389048807 (202h56m58.348146055s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389269495 (202h56m58.348366743s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389303339 (202h56m58.348400583s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389577693 (202h56m58.348674943s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389582267 (202h56m58.348679515s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389868029 (202h56m58.348965273s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389899606 (202h56m58.348996853s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390181444 (202h56m58.349278692s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390456412 (202h56m58.349553657s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.396850594 (202h56m58.355947842s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:37.003486038 (202h56m58.962583287s) [upload=211241]: Next returned nil segment Aug 17 04:22:37.003509888 (202h56m58.962607148s) [upload=211241]: Waiting for error group managing segments... Aug 17 04:22:37.141087302 (202h56m59.100184557s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.147724589 (202h56m59.106821827s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148191395 (202h56m59.107288641s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148249179 (202h56m59.107346437s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.154197920 (202h56m59.113295159s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155088755 (202h56m59.11418599s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155737855 (202h56m59.114835102s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.158542805 (202h56m59.117640044s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.163837059 (202h56m59.122934309s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.225669823 (202h56m59.184767059s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.249261694 (202h56m59.20835894s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:23:05.920185881 (202h57m27.879283118s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: rpc: dial tcp 104.36.82.10:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920277695 (202h57m27.879374941s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: rpc: dial tcp 104.36.82.15:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920476811 (202h57m27.87957406s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: rpc: dial tcp 104.36.82.7:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920550680 (202h57m27.879647915s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920687798 (202h57m27.879785037s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:52446->104.36.82.8:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 04:23:05.920736972 (202h57m27.879834217s) [seg_pos={0 0}, upload=211241]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Aug 17 04:23:06.024578477 (202h57m27.983675715s) [upload=211241]: Done uploading segments... Aug 17 04:23:06.024615107 (202h57m27.983712354s) [upload=211241]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.372490872s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................................XXXX........] duration: 2.455082342s FAIL (2025-08-17 04:28:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277217340 (202h56m58.236314588s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277444018 (202h56m58.236541264s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277693944 (202h56m58.236791191s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277957287 (202h56m58.237054531s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278192843 (202h56m58.237290088s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278363507 (202h56m58.237460782s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278887870 (202h56m58.237985121s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279265735 (202h56m58.238362991s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279639846 (202h56m58.238737091s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279971414 (202h56m58.239068662s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280041108 (202h56m58.239138356s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280352792 (202h56m58.239450046s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280683314 (202h56m58.239780567s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280969649 (202h56m58.240066896s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281241193 (202h56m58.240338438s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281508639 (202h56m58.240605887s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.837376257 (202h56m58.796473489s) [upload=211240]: Next returned nil segment Aug 17 04:22:36.837399552 (202h56m58.796496798s) [upload=211240]: Waiting for error group managing segments... Aug 17 04:22:36.989047654 (202h56m58.948144887s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991378317 (202h56m58.950475552s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991998552 (202h56m58.951095785s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.010955158 (202h56m58.970052395s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.017235751 (202h56m58.97633299s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.045617264 (202h56m59.004714496s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.080535376 (202h56m59.039632621s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.121242355 (202h56m59.080339591s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.296208684 (202h56m59.255305934s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.528205336 (202h56m59.487302581s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:23:35.761418960 (202h57m57.720516213s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1okrDyM4TfzYN3p8zUjTqbw2661rKQQqZdP1ptAKzzTQ82C7cN, address:104.36.82.60:4341): protocol: rpc: dial tcp 104.36.82.60:4341: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761509741 (202h57m57.720606987s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1i62GRFASf7UPVGhUofmRkz3zZSyhq4fDycgyYtfD7pHR3w91v, address:104.36.82.34:4556): protocol: write tcp 139.59.144.22:37150->104.36.82.34:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761574924 (202h57m57.720672174s) [noise=true , address=104.36.82.17:4333 , piece_id=CGHZEP2ZKWES3L6XH4LSFLOFKYLQPYJZ56G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1xRdTZoEk3NsnS445GcdEvwAdDzNLwUYSfgjB7bBSGRx9L9mrU, address:104.36.82.17:4333): protocol: write tcp 139.59.144.22:47980->104.36.82.17:4333: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761620359 (202h57m57.720717605s) [noise=true , address=104.36.82.58:4293 , piece_id=IY3EUM3SZ42OZA42BC2FSVTLTHU5POSMOOQT6MTLHNPZ6FEYE2EQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:15i783VxxzbriD7LZCbFWNBY8uCKUffXtmWE2eu2VupgE1t9bZ, address:104.36.82.58:4293): protocol: write tcp 139.59.144.22:39172->104.36.82.58:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761661063 (202h57m57.720758308s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:12aETKgTzKw8fyYqbeDA2JzfxZnWn12xUPYK1eRt6TvBPPdRJDp, address:104.36.82.36:4630): protocol: write tcp 139.59.144.22:51970->104.36.82.36:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761681516 (202h57m57.720778766s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 139.59.144.22:35702->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761728458 (202h57m57.720825693s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:48846->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761769868 (202h57m57.720867115s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1hbt8HKTunpPKyAzH4XBwGQqTKzcgqPpoWwKZm9JMtaK8P7aVp, address:104.36.82.8:4992): protocol: write tcp 139.59.144.22:40976->104.36.82.8:4992: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761795616 (202h57m57.720892851s) [seg_pos={0 0}, upload=211240]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:35.867721911 (202h57m57.826819144s) [upload=211240]: Done uploading segments... Aug 17 04:23:35.867749737 (202h57m57.826846969s) [upload=211240]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 2.455122004s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................................XXX...........] duration: 2.447592342s FAIL (2025-08-17 04:28:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 17 04:21:58.824378996 (202h56m20.783476241s) [upload=211227]: Starting upload Aug 17 04:21:58.824453728 (202h56m20.783550973s) [upload=211227]: Done starting upload Aug 17 04:21:58.824533712 (202h56m20.783630943s) [upload=211227]: Uploading segments... Aug 17 04:21:58.824814622 (202h56m20.783911863s) [upload=211227]: Got next segment. Inline: false Aug 17 04:21:58.824870724 (202h56m20.78396797s) [upload=211227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:21:58.824876461 (202h56m20.783973706s) [upload=211227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:21:59.025160521 (202h56m20.984257797s) [seg_pos={0 0}, upload=211227]: Begin upload segment... Aug 17 04:21:59.025238019 (202h56m20.984335266s) [seg_pos={0 0}, upload=211227]: Done begin upload segment. Aug 17 04:21:59.025273686 (202h56m20.98437092s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025331026 (202h56m20.984428309s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025756251 (202h56m20.984853496s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025907425 (202h56m20.98500468s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026127117 (202h56m20.98522437s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026350346 (202h56m20.985447594s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026436670 (202h56m20.985533916s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026714369 (202h56m20.985811614s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026788896 (202h56m20.985886144s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027068637 (202h56m20.986165882s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027120600 (202h56m20.986217846s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027420110 (202h56m20.986517355s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027425372 (202h56m20.98652262s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027839122 (202h56m20.986936376s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.028247142 (202h56m20.987344396s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030005732 (202h56m20.989102986s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030432900 (202h56m20.989530147s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.031846491 (202h56m20.990943746s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.588920323 (202h56m21.548017569s) [upload=211227]: Next returned nil segment Aug 17 04:21:59.588944741 (202h56m21.548041972s) [upload=211227]: Waiting for error group managing segments... Aug 17 04:21:59.741262552 (202h56m21.700359791s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.742768290 (202h56m21.701865528s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.777622259 (202h56m21.736719503s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.808692811 (202h56m21.767790057s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.831688332 (202h56m21.790785579s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.871745680 (202h56m21.830842918s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.883314367 (202h56m21.84241162s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.890311438 (202h56m21.849408688s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.940441979 (202h56m21.899539228s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.976656973 (202h56m21.935754218s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.283185203 (202h56m22.242282448s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.284822985 (202h56m22.243920238s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.446988508 (202h56m22.406085753s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:58.545488378 (202h57m20.504585625s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr, address:104.36.82.5:4866): protocol: write tcp 139.59.144.22:57798->104.36.82.5:4866: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545674721 (202h57m20.504771972s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 139.59.144.22:57190->104.36.82.23:4447: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545824338 (202h57m20.504921585s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1, address:104.36.82.44:4901): protocol: write tcp 139.59.144.22:54098->104.36.82.44:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545853626 (202h57m20.504950858s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12ASAeTMBu4sY1GuZiSuwTXmhPksA1Te2BVZdcoE9QvpLAmcQX3, address:104.36.82.35:4603): protocol: write tcp 139.59.144.22:49164->104.36.82.35:4603: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545907146 (202h57m20.505004393s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: write tcp 139.59.144.22:43990->104.36.82.60:4365: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545989950 (202h57m20.505087195s) [seg_pos={0 0}, upload=211227]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 04:22:58.646357515 (202h57m20.605454767s) [upload=211227]: Done uploading segments... Aug 17 04:22:58.646391150 (202h57m20.605488398s) [upload=211227]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.447638004s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..................................................XXXXX.....] duration: 2.811229282s FAIL (2025-08-17 04:27:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.811254273s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................................XXX..........] duration: 2.472074206s FAIL (2025-08-17 04:27:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 17 04:21:58.824378996 (202h56m20.783476241s) [upload=211227]: Starting upload Aug 17 04:21:58.824453728 (202h56m20.783550973s) [upload=211227]: Done starting upload Aug 17 04:21:58.824533712 (202h56m20.783630943s) [upload=211227]: Uploading segments... Aug 17 04:21:58.824814622 (202h56m20.783911863s) [upload=211227]: Got next segment. Inline: false Aug 17 04:21:58.824870724 (202h56m20.78396797s) [upload=211227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:21:58.824876461 (202h56m20.783973706s) [upload=211227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:21:59.025160521 (202h56m20.984257797s) [seg_pos={0 0}, upload=211227]: Begin upload segment... Aug 17 04:21:59.025238019 (202h56m20.984335266s) [seg_pos={0 0}, upload=211227]: Done begin upload segment. Aug 17 04:21:59.025273686 (202h56m20.98437092s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025331026 (202h56m20.984428309s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025756251 (202h56m20.984853496s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025907425 (202h56m20.98500468s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026127117 (202h56m20.98522437s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026350346 (202h56m20.985447594s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026436670 (202h56m20.985533916s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026714369 (202h56m20.985811614s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026788896 (202h56m20.985886144s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027068637 (202h56m20.986165882s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027120600 (202h56m20.986217846s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027420110 (202h56m20.986517355s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027425372 (202h56m20.98652262s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027839122 (202h56m20.986936376s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.028247142 (202h56m20.987344396s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030005732 (202h56m20.989102986s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030432900 (202h56m20.989530147s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.031846491 (202h56m20.990943746s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.588920323 (202h56m21.548017569s) [upload=211227]: Next returned nil segment Aug 17 04:21:59.588944741 (202h56m21.548041972s) [upload=211227]: Waiting for error group managing segments... Aug 17 04:21:59.741262552 (202h56m21.700359791s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.742768290 (202h56m21.701865528s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.777622259 (202h56m21.736719503s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.808692811 (202h56m21.767790057s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.831688332 (202h56m21.790785579s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.871745680 (202h56m21.830842918s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.883314367 (202h56m21.84241162s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.890311438 (202h56m21.849408688s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.940441979 (202h56m21.899539228s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.976656973 (202h56m21.935754218s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.283185203 (202h56m22.242282448s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.284822985 (202h56m22.243920238s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.446988508 (202h56m22.406085753s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:58.545488378 (202h57m20.504585625s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr, address:104.36.82.5:4866): protocol: write tcp 139.59.144.22:57798->104.36.82.5:4866: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545674721 (202h57m20.504771972s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 139.59.144.22:57190->104.36.82.23:4447: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545824338 (202h57m20.504921585s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1, address:104.36.82.44:4901): protocol: write tcp 139.59.144.22:54098->104.36.82.44:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545853626 (202h57m20.504950858s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12ASAeTMBu4sY1GuZiSuwTXmhPksA1Te2BVZdcoE9QvpLAmcQX3, address:104.36.82.35:4603): protocol: write tcp 139.59.144.22:49164->104.36.82.35:4603: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545907146 (202h57m20.505004393s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: write tcp 139.59.144.22:43990->104.36.82.60:4365: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545989950 (202h57m20.505087195s) [seg_pos={0 0}, upload=211227]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 04:22:58.646357515 (202h57m20.605454767s) [upload=211227]: Done uploading segments... Aug 17 04:22:58.646391150 (202h57m20.605488398s) [upload=211227]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.472141869s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..................................................XX........] duration: 3.015121865s FAIL (2025-08-17 04:27:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.015151932s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................................XXXX.....] duration: 2.441423003s FAIL (2025-08-17 04:27:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Aug 17 04:22:36.193175039 (202h56m58.152272297s) [upload=211241]: Starting upload Aug 17 04:22:36.193282733 (202h56m58.152379987s) [upload=211241]: Done starting upload Aug 17 04:22:36.193533603 (202h56m58.152630849s) [upload=211241]: Uploading segments... Aug 17 04:22:36.193824638 (202h56m58.152921893s) [upload=211241]: Got next segment. Inline: false Aug 17 04:22:36.193891646 (202h56m58.152988897s) [upload=211241]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:22:36.193898487 (202h56m58.15299574s) [upload=211241]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:22:36.387313731 (202h56m58.346410984s) [seg_pos={0 0}, upload=211241]: Begin upload segment... Aug 17 04:22:36.387391508 (202h56m58.346488754s) [seg_pos={0 0}, upload=211241]: Done begin upload segment. Aug 17 04:22:36.387453494 (202h56m58.34655085s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.387549035 (202h56m58.346646289s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388292176 (202h56m58.347389421s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388564570 (202h56m58.347661824s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388775483 (202h56m58.347872728s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388976887 (202h56m58.348074132s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389048807 (202h56m58.348146055s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389269495 (202h56m58.348366743s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389303339 (202h56m58.348400583s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389577693 (202h56m58.348674943s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389582267 (202h56m58.348679515s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389868029 (202h56m58.348965273s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389899606 (202h56m58.348996853s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390181444 (202h56m58.349278692s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390456412 (202h56m58.349553657s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.396850594 (202h56m58.355947842s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:37.003486038 (202h56m58.962583287s) [upload=211241]: Next returned nil segment Aug 17 04:22:37.003509888 (202h56m58.962607148s) [upload=211241]: Waiting for error group managing segments... Aug 17 04:22:37.141087302 (202h56m59.100184557s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.147724589 (202h56m59.106821827s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148191395 (202h56m59.107288641s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148249179 (202h56m59.107346437s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.154197920 (202h56m59.113295159s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155088755 (202h56m59.11418599s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155737855 (202h56m59.114835102s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.158542805 (202h56m59.117640044s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.163837059 (202h56m59.122934309s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.225669823 (202h56m59.184767059s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.249261694 (202h56m59.20835894s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:23:05.920185881 (202h57m27.879283118s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: rpc: dial tcp 104.36.82.10:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920277695 (202h57m27.879374941s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: rpc: dial tcp 104.36.82.15:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920476811 (202h57m27.87957406s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: rpc: dial tcp 104.36.82.7:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920550680 (202h57m27.879647915s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920687798 (202h57m27.879785037s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:52446->104.36.82.8:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 04:23:05.920736972 (202h57m27.879834217s) [seg_pos={0 0}, upload=211241]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Aug 17 04:23:06.024578477 (202h57m27.983675715s) [upload=211241]: Done uploading segments... Aug 17 04:23:06.024615107 (202h57m27.983712354s) [upload=211241]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.441457842s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................................XXXX.......] duration: 2.696297562s FAIL (2025-08-17 04:27:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277217340 (202h56m58.236314588s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277444018 (202h56m58.236541264s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277693944 (202h56m58.236791191s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277957287 (202h56m58.237054531s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278192843 (202h56m58.237290088s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278363507 (202h56m58.237460782s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278887870 (202h56m58.237985121s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279265735 (202h56m58.238362991s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279639846 (202h56m58.238737091s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279971414 (202h56m58.239068662s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280041108 (202h56m58.239138356s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280352792 (202h56m58.239450046s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280683314 (202h56m58.239780567s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280969649 (202h56m58.240066896s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281241193 (202h56m58.240338438s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281508639 (202h56m58.240605887s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.837376257 (202h56m58.796473489s) [upload=211240]: Next returned nil segment Aug 17 04:22:36.837399552 (202h56m58.796496798s) [upload=211240]: Waiting for error group managing segments... Aug 17 04:22:36.989047654 (202h56m58.948144887s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991378317 (202h56m58.950475552s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991998552 (202h56m58.951095785s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.010955158 (202h56m58.970052395s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.017235751 (202h56m58.97633299s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.045617264 (202h56m59.004714496s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.080535376 (202h56m59.039632621s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.121242355 (202h56m59.080339591s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.296208684 (202h56m59.255305934s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.528205336 (202h56m59.487302581s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:23:35.761418960 (202h57m57.720516213s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1okrDyM4TfzYN3p8zUjTqbw2661rKQQqZdP1ptAKzzTQ82C7cN, address:104.36.82.60:4341): protocol: rpc: dial tcp 104.36.82.60:4341: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761509741 (202h57m57.720606987s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1i62GRFASf7UPVGhUofmRkz3zZSyhq4fDycgyYtfD7pHR3w91v, address:104.36.82.34:4556): protocol: write tcp 139.59.144.22:37150->104.36.82.34:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761574924 (202h57m57.720672174s) [noise=true , address=104.36.82.17:4333 , piece_id=CGHZEP2ZKWES3L6XH4LSFLOFKYLQPYJZ56G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1xRdTZoEk3NsnS445GcdEvwAdDzNLwUYSfgjB7bBSGRx9L9mrU, address:104.36.82.17:4333): protocol: write tcp 139.59.144.22:47980->104.36.82.17:4333: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761620359 (202h57m57.720717605s) [noise=true , address=104.36.82.58:4293 , piece_id=IY3EUM3SZ42OZA42BC2FSVTLTHU5POSMOOQT6MTLHNPZ6FEYE2EQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:15i783VxxzbriD7LZCbFWNBY8uCKUffXtmWE2eu2VupgE1t9bZ, address:104.36.82.58:4293): protocol: write tcp 139.59.144.22:39172->104.36.82.58:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761661063 (202h57m57.720758308s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:12aETKgTzKw8fyYqbeDA2JzfxZnWn12xUPYK1eRt6TvBPPdRJDp, address:104.36.82.36:4630): protocol: write tcp 139.59.144.22:51970->104.36.82.36:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761681516 (202h57m57.720778766s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 139.59.144.22:35702->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761728458 (202h57m57.720825693s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:48846->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761769868 (202h57m57.720867115s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1hbt8HKTunpPKyAzH4XBwGQqTKzcgqPpoWwKZm9JMtaK8P7aVp, address:104.36.82.8:4992): protocol: write tcp 139.59.144.22:40976->104.36.82.8:4992: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761795616 (202h57m57.720892851s) [seg_pos={0 0}, upload=211240]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:35.867721911 (202h57m57.826819144s) [upload=211240]: Done uploading segments... Aug 17 04:23:35.867749737 (202h57m57.826846969s) [upload=211240]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 2.696402785s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...................................................XX.......] duration: 3.800540222s FAIL (2025-08-17 04:27:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.800575349s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................................XXX.........] duration: 2.419974384s FAIL (2025-08-17 04:27:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 17 04:21:58.824378996 (202h56m20.783476241s) [upload=211227]: Starting upload Aug 17 04:21:58.824453728 (202h56m20.783550973s) [upload=211227]: Done starting upload Aug 17 04:21:58.824533712 (202h56m20.783630943s) [upload=211227]: Uploading segments... Aug 17 04:21:58.824814622 (202h56m20.783911863s) [upload=211227]: Got next segment. Inline: false Aug 17 04:21:58.824870724 (202h56m20.78396797s) [upload=211227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:21:58.824876461 (202h56m20.783973706s) [upload=211227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:21:59.025160521 (202h56m20.984257797s) [seg_pos={0 0}, upload=211227]: Begin upload segment... Aug 17 04:21:59.025238019 (202h56m20.984335266s) [seg_pos={0 0}, upload=211227]: Done begin upload segment. Aug 17 04:21:59.025273686 (202h56m20.98437092s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025331026 (202h56m20.984428309s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025756251 (202h56m20.984853496s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025907425 (202h56m20.98500468s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026127117 (202h56m20.98522437s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026350346 (202h56m20.985447594s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026436670 (202h56m20.985533916s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026714369 (202h56m20.985811614s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026788896 (202h56m20.985886144s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027068637 (202h56m20.986165882s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027120600 (202h56m20.986217846s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027420110 (202h56m20.986517355s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027425372 (202h56m20.98652262s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027839122 (202h56m20.986936376s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.028247142 (202h56m20.987344396s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030005732 (202h56m20.989102986s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030432900 (202h56m20.989530147s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.031846491 (202h56m20.990943746s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.588920323 (202h56m21.548017569s) [upload=211227]: Next returned nil segment Aug 17 04:21:59.588944741 (202h56m21.548041972s) [upload=211227]: Waiting for error group managing segments... Aug 17 04:21:59.741262552 (202h56m21.700359791s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.742768290 (202h56m21.701865528s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.777622259 (202h56m21.736719503s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.808692811 (202h56m21.767790057s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.831688332 (202h56m21.790785579s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.871745680 (202h56m21.830842918s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.883314367 (202h56m21.84241162s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.890311438 (202h56m21.849408688s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.940441979 (202h56m21.899539228s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.976656973 (202h56m21.935754218s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.283185203 (202h56m22.242282448s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.284822985 (202h56m22.243920238s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.446988508 (202h56m22.406085753s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:58.545488378 (202h57m20.504585625s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr, address:104.36.82.5:4866): protocol: write tcp 139.59.144.22:57798->104.36.82.5:4866: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545674721 (202h57m20.504771972s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 139.59.144.22:57190->104.36.82.23:4447: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545824338 (202h57m20.504921585s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1, address:104.36.82.44:4901): protocol: write tcp 139.59.144.22:54098->104.36.82.44:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545853626 (202h57m20.504950858s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12ASAeTMBu4sY1GuZiSuwTXmhPksA1Te2BVZdcoE9QvpLAmcQX3, address:104.36.82.35:4603): protocol: write tcp 139.59.144.22:49164->104.36.82.35:4603: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545907146 (202h57m20.505004393s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: write tcp 139.59.144.22:43990->104.36.82.60:4365: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545989950 (202h57m20.505087195s) [seg_pos={0 0}, upload=211227]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 04:22:58.646357515 (202h57m20.605454767s) [upload=211227]: Done uploading segments... Aug 17 04:22:58.646391150 (202h57m20.605488398s) [upload=211227]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.420009904s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...................................................XXXXX....] duration: 2.907195388s FAIL (2025-08-17 04:27:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.907221825s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................................XXX........] duration: 3.020302378s FAIL (2025-08-17 04:26:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 17 04:21:58.824378996 (202h56m20.783476241s) [upload=211227]: Starting upload Aug 17 04:21:58.824453728 (202h56m20.783550973s) [upload=211227]: Done starting upload Aug 17 04:21:58.824533712 (202h56m20.783630943s) [upload=211227]: Uploading segments... Aug 17 04:21:58.824814622 (202h56m20.783911863s) [upload=211227]: Got next segment. Inline: false Aug 17 04:21:58.824870724 (202h56m20.78396797s) [upload=211227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:21:58.824876461 (202h56m20.783973706s) [upload=211227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:21:59.025160521 (202h56m20.984257797s) [seg_pos={0 0}, upload=211227]: Begin upload segment... Aug 17 04:21:59.025238019 (202h56m20.984335266s) [seg_pos={0 0}, upload=211227]: Done begin upload segment. Aug 17 04:21:59.025273686 (202h56m20.98437092s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025331026 (202h56m20.984428309s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025756251 (202h56m20.984853496s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025907425 (202h56m20.98500468s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026127117 (202h56m20.98522437s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026350346 (202h56m20.985447594s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026436670 (202h56m20.985533916s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026714369 (202h56m20.985811614s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026788896 (202h56m20.985886144s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027068637 (202h56m20.986165882s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027120600 (202h56m20.986217846s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027420110 (202h56m20.986517355s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027425372 (202h56m20.98652262s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027839122 (202h56m20.986936376s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.028247142 (202h56m20.987344396s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030005732 (202h56m20.989102986s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030432900 (202h56m20.989530147s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.031846491 (202h56m20.990943746s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.588920323 (202h56m21.548017569s) [upload=211227]: Next returned nil segment Aug 17 04:21:59.588944741 (202h56m21.548041972s) [upload=211227]: Waiting for error group managing segments... Aug 17 04:21:59.741262552 (202h56m21.700359791s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.742768290 (202h56m21.701865528s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.777622259 (202h56m21.736719503s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.808692811 (202h56m21.767790057s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.831688332 (202h56m21.790785579s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.871745680 (202h56m21.830842918s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.883314367 (202h56m21.84241162s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.890311438 (202h56m21.849408688s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.940441979 (202h56m21.899539228s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.976656973 (202h56m21.935754218s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.283185203 (202h56m22.242282448s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.284822985 (202h56m22.243920238s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.446988508 (202h56m22.406085753s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:58.545488378 (202h57m20.504585625s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr, address:104.36.82.5:4866): protocol: write tcp 139.59.144.22:57798->104.36.82.5:4866: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545674721 (202h57m20.504771972s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 139.59.144.22:57190->104.36.82.23:4447: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545824338 (202h57m20.504921585s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1, address:104.36.82.44:4901): protocol: write tcp 139.59.144.22:54098->104.36.82.44:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545853626 (202h57m20.504950858s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12ASAeTMBu4sY1GuZiSuwTXmhPksA1Te2BVZdcoE9QvpLAmcQX3, address:104.36.82.35:4603): protocol: write tcp 139.59.144.22:49164->104.36.82.35:4603: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545907146 (202h57m20.505004393s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: write tcp 139.59.144.22:43990->104.36.82.60:4365: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545989950 (202h57m20.505087195s) [seg_pos={0 0}, upload=211227]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 04:22:58.646357515 (202h57m20.605454767s) [upload=211227]: Done uploading segments... Aug 17 04:22:58.646391150 (202h57m20.605488398s) [upload=211227]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.020335979s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................................................XXXX....] duration: 2.4180607s FAIL (2025-08-17 04:26:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Aug 17 04:22:36.193175039 (202h56m58.152272297s) [upload=211241]: Starting upload Aug 17 04:22:36.193282733 (202h56m58.152379987s) [upload=211241]: Done starting upload Aug 17 04:22:36.193533603 (202h56m58.152630849s) [upload=211241]: Uploading segments... Aug 17 04:22:36.193824638 (202h56m58.152921893s) [upload=211241]: Got next segment. Inline: false Aug 17 04:22:36.193891646 (202h56m58.152988897s) [upload=211241]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:22:36.193898487 (202h56m58.15299574s) [upload=211241]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:22:36.387313731 (202h56m58.346410984s) [seg_pos={0 0}, upload=211241]: Begin upload segment... Aug 17 04:22:36.387391508 (202h56m58.346488754s) [seg_pos={0 0}, upload=211241]: Done begin upload segment. Aug 17 04:22:36.387453494 (202h56m58.34655085s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.387549035 (202h56m58.346646289s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388292176 (202h56m58.347389421s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388564570 (202h56m58.347661824s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388775483 (202h56m58.347872728s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388976887 (202h56m58.348074132s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389048807 (202h56m58.348146055s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389269495 (202h56m58.348366743s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389303339 (202h56m58.348400583s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389577693 (202h56m58.348674943s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389582267 (202h56m58.348679515s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389868029 (202h56m58.348965273s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389899606 (202h56m58.348996853s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390181444 (202h56m58.349278692s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390456412 (202h56m58.349553657s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.396850594 (202h56m58.355947842s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:37.003486038 (202h56m58.962583287s) [upload=211241]: Next returned nil segment Aug 17 04:22:37.003509888 (202h56m58.962607148s) [upload=211241]: Waiting for error group managing segments... Aug 17 04:22:37.141087302 (202h56m59.100184557s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.147724589 (202h56m59.106821827s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148191395 (202h56m59.107288641s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148249179 (202h56m59.107346437s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.154197920 (202h56m59.113295159s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155088755 (202h56m59.11418599s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155737855 (202h56m59.114835102s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.158542805 (202h56m59.117640044s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.163837059 (202h56m59.122934309s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.225669823 (202h56m59.184767059s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.249261694 (202h56m59.20835894s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:23:05.920185881 (202h57m27.879283118s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: rpc: dial tcp 104.36.82.10:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920277695 (202h57m27.879374941s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: rpc: dial tcp 104.36.82.15:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920476811 (202h57m27.87957406s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: rpc: dial tcp 104.36.82.7:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920550680 (202h57m27.879647915s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920687798 (202h57m27.879785037s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:52446->104.36.82.8:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 04:23:05.920736972 (202h57m27.879834217s) [seg_pos={0 0}, upload=211241]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Aug 17 04:23:06.024578477 (202h57m27.983675715s) [upload=211241]: Done uploading segments... Aug 17 04:23:06.024615107 (202h57m27.983712354s) [upload=211241]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.418112526s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................................XXX.......] duration: 2.597810649s FAIL (2025-08-17 04:26:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 17 04:21:58.824378996 (202h56m20.783476241s) [upload=211227]: Starting upload Aug 17 04:21:58.824453728 (202h56m20.783550973s) [upload=211227]: Done starting upload Aug 17 04:21:58.824533712 (202h56m20.783630943s) [upload=211227]: Uploading segments... Aug 17 04:21:58.824814622 (202h56m20.783911863s) [upload=211227]: Got next segment. Inline: false Aug 17 04:21:58.824870724 (202h56m20.78396797s) [upload=211227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:21:58.824876461 (202h56m20.783973706s) [upload=211227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:21:59.025160521 (202h56m20.984257797s) [seg_pos={0 0}, upload=211227]: Begin upload segment... Aug 17 04:21:59.025238019 (202h56m20.984335266s) [seg_pos={0 0}, upload=211227]: Done begin upload segment. Aug 17 04:21:59.025273686 (202h56m20.98437092s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025331026 (202h56m20.984428309s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025756251 (202h56m20.984853496s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025907425 (202h56m20.98500468s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026127117 (202h56m20.98522437s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026350346 (202h56m20.985447594s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026436670 (202h56m20.985533916s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026714369 (202h56m20.985811614s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026788896 (202h56m20.985886144s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027068637 (202h56m20.986165882s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027120600 (202h56m20.986217846s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027420110 (202h56m20.986517355s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027425372 (202h56m20.98652262s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027839122 (202h56m20.986936376s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.028247142 (202h56m20.987344396s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030005732 (202h56m20.989102986s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030432900 (202h56m20.989530147s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.031846491 (202h56m20.990943746s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.588920323 (202h56m21.548017569s) [upload=211227]: Next returned nil segment Aug 17 04:21:59.588944741 (202h56m21.548041972s) [upload=211227]: Waiting for error group managing segments... Aug 17 04:21:59.741262552 (202h56m21.700359791s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.742768290 (202h56m21.701865528s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.777622259 (202h56m21.736719503s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.808692811 (202h56m21.767790057s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.831688332 (202h56m21.790785579s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.871745680 (202h56m21.830842918s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.883314367 (202h56m21.84241162s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.890311438 (202h56m21.849408688s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.940441979 (202h56m21.899539228s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.976656973 (202h56m21.935754218s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.283185203 (202h56m22.242282448s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.284822985 (202h56m22.243920238s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.446988508 (202h56m22.406085753s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:58.545488378 (202h57m20.504585625s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr, address:104.36.82.5:4866): protocol: write tcp 139.59.144.22:57798->104.36.82.5:4866: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545674721 (202h57m20.504771972s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 139.59.144.22:57190->104.36.82.23:4447: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545824338 (202h57m20.504921585s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1, address:104.36.82.44:4901): protocol: write tcp 139.59.144.22:54098->104.36.82.44:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545853626 (202h57m20.504950858s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12ASAeTMBu4sY1GuZiSuwTXmhPksA1Te2BVZdcoE9QvpLAmcQX3, address:104.36.82.35:4603): protocol: write tcp 139.59.144.22:49164->104.36.82.35:4603: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545907146 (202h57m20.505004393s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: write tcp 139.59.144.22:43990->104.36.82.60:4365: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545989950 (202h57m20.505087195s) [seg_pos={0 0}, upload=211227]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 04:22:58.646357515 (202h57m20.605454767s) [upload=211227]: Done uploading segments... Aug 17 04:22:58.646391150 (202h57m20.605488398s) [upload=211227]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.597859652s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................................XXXX......] duration: 3.20080499s FAIL (2025-08-17 04:26:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277217340 (202h56m58.236314588s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277444018 (202h56m58.236541264s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277693944 (202h56m58.236791191s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277957287 (202h56m58.237054531s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278192843 (202h56m58.237290088s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278363507 (202h56m58.237460782s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278887870 (202h56m58.237985121s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279265735 (202h56m58.238362991s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279639846 (202h56m58.238737091s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279971414 (202h56m58.239068662s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280041108 (202h56m58.239138356s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280352792 (202h56m58.239450046s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280683314 (202h56m58.239780567s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280969649 (202h56m58.240066896s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281241193 (202h56m58.240338438s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281508639 (202h56m58.240605887s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.837376257 (202h56m58.796473489s) [upload=211240]: Next returned nil segment Aug 17 04:22:36.837399552 (202h56m58.796496798s) [upload=211240]: Waiting for error group managing segments... Aug 17 04:22:36.989047654 (202h56m58.948144887s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991378317 (202h56m58.950475552s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991998552 (202h56m58.951095785s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.010955158 (202h56m58.970052395s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.017235751 (202h56m58.97633299s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.045617264 (202h56m59.004714496s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.080535376 (202h56m59.039632621s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.121242355 (202h56m59.080339591s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.296208684 (202h56m59.255305934s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.528205336 (202h56m59.487302581s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:23:35.761418960 (202h57m57.720516213s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1okrDyM4TfzYN3p8zUjTqbw2661rKQQqZdP1ptAKzzTQ82C7cN, address:104.36.82.60:4341): protocol: rpc: dial tcp 104.36.82.60:4341: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761509741 (202h57m57.720606987s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1i62GRFASf7UPVGhUofmRkz3zZSyhq4fDycgyYtfD7pHR3w91v, address:104.36.82.34:4556): protocol: write tcp 139.59.144.22:37150->104.36.82.34:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761574924 (202h57m57.720672174s) [noise=true , address=104.36.82.17:4333 , piece_id=CGHZEP2ZKWES3L6XH4LSFLOFKYLQPYJZ56G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1xRdTZoEk3NsnS445GcdEvwAdDzNLwUYSfgjB7bBSGRx9L9mrU, address:104.36.82.17:4333): protocol: write tcp 139.59.144.22:47980->104.36.82.17:4333: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761620359 (202h57m57.720717605s) [noise=true , address=104.36.82.58:4293 , piece_id=IY3EUM3SZ42OZA42BC2FSVTLTHU5POSMOOQT6MTLHNPZ6FEYE2EQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:15i783VxxzbriD7LZCbFWNBY8uCKUffXtmWE2eu2VupgE1t9bZ, address:104.36.82.58:4293): protocol: write tcp 139.59.144.22:39172->104.36.82.58:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761661063 (202h57m57.720758308s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:12aETKgTzKw8fyYqbeDA2JzfxZnWn12xUPYK1eRt6TvBPPdRJDp, address:104.36.82.36:4630): protocol: write tcp 139.59.144.22:51970->104.36.82.36:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761681516 (202h57m57.720778766s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 139.59.144.22:35702->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761728458 (202h57m57.720825693s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:48846->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761769868 (202h57m57.720867115s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1hbt8HKTunpPKyAzH4XBwGQqTKzcgqPpoWwKZm9JMtaK8P7aVp, address:104.36.82.8:4992): protocol: write tcp 139.59.144.22:40976->104.36.82.8:4992: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761795616 (202h57m57.720892851s) [seg_pos={0 0}, upload=211240]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:35.867721911 (202h57m57.826819144s) [upload=211240]: Done uploading segments... Aug 17 04:23:35.867749737 (202h57m57.826846969s) [upload=211240]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 3.200849187s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....................................................XX......] duration: 3.259250494s FAIL (2025-08-17 04:26:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.259287598s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [....................................................XXXXX...] duration: 2.762225188s FAIL (2025-08-17 04:26:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.762252277s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................................XXX......] duration: 2.67807988s FAIL (2025-08-17 04:26:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 17 04:21:58.824378996 (202h56m20.783476241s) [upload=211227]: Starting upload Aug 17 04:21:58.824453728 (202h56m20.783550973s) [upload=211227]: Done starting upload Aug 17 04:21:58.824533712 (202h56m20.783630943s) [upload=211227]: Uploading segments... Aug 17 04:21:58.824814622 (202h56m20.783911863s) [upload=211227]: Got next segment. Inline: false Aug 17 04:21:58.824870724 (202h56m20.78396797s) [upload=211227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:21:58.824876461 (202h56m20.783973706s) [upload=211227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:21:59.025160521 (202h56m20.984257797s) [seg_pos={0 0}, upload=211227]: Begin upload segment... Aug 17 04:21:59.025238019 (202h56m20.984335266s) [seg_pos={0 0}, upload=211227]: Done begin upload segment. Aug 17 04:21:59.025273686 (202h56m20.98437092s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025331026 (202h56m20.984428309s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025756251 (202h56m20.984853496s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025907425 (202h56m20.98500468s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026127117 (202h56m20.98522437s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026350346 (202h56m20.985447594s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026436670 (202h56m20.985533916s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026714369 (202h56m20.985811614s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026788896 (202h56m20.985886144s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027068637 (202h56m20.986165882s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027120600 (202h56m20.986217846s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027420110 (202h56m20.986517355s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027425372 (202h56m20.98652262s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027839122 (202h56m20.986936376s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.028247142 (202h56m20.987344396s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030005732 (202h56m20.989102986s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030432900 (202h56m20.989530147s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.031846491 (202h56m20.990943746s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.588920323 (202h56m21.548017569s) [upload=211227]: Next returned nil segment Aug 17 04:21:59.588944741 (202h56m21.548041972s) [upload=211227]: Waiting for error group managing segments... Aug 17 04:21:59.741262552 (202h56m21.700359791s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.742768290 (202h56m21.701865528s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.777622259 (202h56m21.736719503s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.808692811 (202h56m21.767790057s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.831688332 (202h56m21.790785579s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.871745680 (202h56m21.830842918s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.883314367 (202h56m21.84241162s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.890311438 (202h56m21.849408688s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.940441979 (202h56m21.899539228s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.976656973 (202h56m21.935754218s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.283185203 (202h56m22.242282448s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.284822985 (202h56m22.243920238s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.446988508 (202h56m22.406085753s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:58.545488378 (202h57m20.504585625s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr, address:104.36.82.5:4866): protocol: write tcp 139.59.144.22:57798->104.36.82.5:4866: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545674721 (202h57m20.504771972s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 139.59.144.22:57190->104.36.82.23:4447: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545824338 (202h57m20.504921585s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1, address:104.36.82.44:4901): protocol: write tcp 139.59.144.22:54098->104.36.82.44:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545853626 (202h57m20.504950858s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12ASAeTMBu4sY1GuZiSuwTXmhPksA1Te2BVZdcoE9QvpLAmcQX3, address:104.36.82.35:4603): protocol: write tcp 139.59.144.22:49164->104.36.82.35:4603: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545907146 (202h57m20.505004393s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: write tcp 139.59.144.22:43990->104.36.82.60:4365: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545989950 (202h57m20.505087195s) [seg_pos={0 0}, upload=211227]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 04:22:58.646357515 (202h57m20.605454767s) [upload=211227]: Done uploading segments... Aug 17 04:22:58.646391150 (202h57m20.605488398s) [upload=211227]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.67811514s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................................XXXX.....] duration: 2.332848923s FAIL (2025-08-17 04:25:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277217340 (202h56m58.236314588s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277444018 (202h56m58.236541264s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277693944 (202h56m58.236791191s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277957287 (202h56m58.237054531s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278192843 (202h56m58.237290088s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278363507 (202h56m58.237460782s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278887870 (202h56m58.237985121s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279265735 (202h56m58.238362991s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279639846 (202h56m58.238737091s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279971414 (202h56m58.239068662s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280041108 (202h56m58.239138356s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280352792 (202h56m58.239450046s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280683314 (202h56m58.239780567s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280969649 (202h56m58.240066896s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281241193 (202h56m58.240338438s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281508639 (202h56m58.240605887s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.837376257 (202h56m58.796473489s) [upload=211240]: Next returned nil segment Aug 17 04:22:36.837399552 (202h56m58.796496798s) [upload=211240]: Waiting for error group managing segments... Aug 17 04:22:36.989047654 (202h56m58.948144887s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991378317 (202h56m58.950475552s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991998552 (202h56m58.951095785s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.010955158 (202h56m58.970052395s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.017235751 (202h56m58.97633299s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.045617264 (202h56m59.004714496s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.080535376 (202h56m59.039632621s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.121242355 (202h56m59.080339591s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.296208684 (202h56m59.255305934s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.528205336 (202h56m59.487302581s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:23:35.761418960 (202h57m57.720516213s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1okrDyM4TfzYN3p8zUjTqbw2661rKQQqZdP1ptAKzzTQ82C7cN, address:104.36.82.60:4341): protocol: rpc: dial tcp 104.36.82.60:4341: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761509741 (202h57m57.720606987s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1i62GRFASf7UPVGhUofmRkz3zZSyhq4fDycgyYtfD7pHR3w91v, address:104.36.82.34:4556): protocol: write tcp 139.59.144.22:37150->104.36.82.34:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761574924 (202h57m57.720672174s) [noise=true , address=104.36.82.17:4333 , piece_id=CGHZEP2ZKWES3L6XH4LSFLOFKYLQPYJZ56G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1xRdTZoEk3NsnS445GcdEvwAdDzNLwUYSfgjB7bBSGRx9L9mrU, address:104.36.82.17:4333): protocol: write tcp 139.59.144.22:47980->104.36.82.17:4333: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761620359 (202h57m57.720717605s) [noise=true , address=104.36.82.58:4293 , piece_id=IY3EUM3SZ42OZA42BC2FSVTLTHU5POSMOOQT6MTLHNPZ6FEYE2EQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:15i783VxxzbriD7LZCbFWNBY8uCKUffXtmWE2eu2VupgE1t9bZ, address:104.36.82.58:4293): protocol: write tcp 139.59.144.22:39172->104.36.82.58:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761661063 (202h57m57.720758308s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:12aETKgTzKw8fyYqbeDA2JzfxZnWn12xUPYK1eRt6TvBPPdRJDp, address:104.36.82.36:4630): protocol: write tcp 139.59.144.22:51970->104.36.82.36:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761681516 (202h57m57.720778766s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 139.59.144.22:35702->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761728458 (202h57m57.720825693s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:48846->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761769868 (202h57m57.720867115s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1hbt8HKTunpPKyAzH4XBwGQqTKzcgqPpoWwKZm9JMtaK8P7aVp, address:104.36.82.8:4992): protocol: write tcp 139.59.144.22:40976->104.36.82.8:4992: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761795616 (202h57m57.720892851s) [seg_pos={0 0}, upload=211240]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:35.867721911 (202h57m57.826819144s) [upload=211240]: Done uploading segments... Aug 17 04:23:35.867749737 (202h57m57.826846969s) [upload=211240]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 2.332888504s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................................................XXXX...] duration: 2.443090972s FAIL (2025-08-17 04:25:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Aug 17 04:22:36.193175039 (202h56m58.152272297s) [upload=211241]: Starting upload Aug 17 04:22:36.193282733 (202h56m58.152379987s) [upload=211241]: Done starting upload Aug 17 04:22:36.193533603 (202h56m58.152630849s) [upload=211241]: Uploading segments... Aug 17 04:22:36.193824638 (202h56m58.152921893s) [upload=211241]: Got next segment. Inline: false Aug 17 04:22:36.193891646 (202h56m58.152988897s) [upload=211241]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:22:36.193898487 (202h56m58.15299574s) [upload=211241]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:22:36.387313731 (202h56m58.346410984s) [seg_pos={0 0}, upload=211241]: Begin upload segment... Aug 17 04:22:36.387391508 (202h56m58.346488754s) [seg_pos={0 0}, upload=211241]: Done begin upload segment. Aug 17 04:22:36.387453494 (202h56m58.34655085s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.387549035 (202h56m58.346646289s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388292176 (202h56m58.347389421s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388564570 (202h56m58.347661824s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388775483 (202h56m58.347872728s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388976887 (202h56m58.348074132s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389048807 (202h56m58.348146055s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389269495 (202h56m58.348366743s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389303339 (202h56m58.348400583s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389577693 (202h56m58.348674943s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389582267 (202h56m58.348679515s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389868029 (202h56m58.348965273s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389899606 (202h56m58.348996853s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390181444 (202h56m58.349278692s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390456412 (202h56m58.349553657s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.396850594 (202h56m58.355947842s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:37.003486038 (202h56m58.962583287s) [upload=211241]: Next returned nil segment Aug 17 04:22:37.003509888 (202h56m58.962607148s) [upload=211241]: Waiting for error group managing segments... Aug 17 04:22:37.141087302 (202h56m59.100184557s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.147724589 (202h56m59.106821827s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148191395 (202h56m59.107288641s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148249179 (202h56m59.107346437s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.154197920 (202h56m59.113295159s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155088755 (202h56m59.11418599s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155737855 (202h56m59.114835102s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.158542805 (202h56m59.117640044s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.163837059 (202h56m59.122934309s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.225669823 (202h56m59.184767059s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.249261694 (202h56m59.20835894s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:23:05.920185881 (202h57m27.879283118s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: rpc: dial tcp 104.36.82.10:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920277695 (202h57m27.879374941s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: rpc: dial tcp 104.36.82.15:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920476811 (202h57m27.87957406s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: rpc: dial tcp 104.36.82.7:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920550680 (202h57m27.879647915s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920687798 (202h57m27.879785037s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:52446->104.36.82.8:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 04:23:05.920736972 (202h57m27.879834217s) [seg_pos={0 0}, upload=211241]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Aug 17 04:23:06.024578477 (202h57m27.983675715s) [upload=211241]: Done uploading segments... Aug 17 04:23:06.024615107 (202h57m27.983712354s) [upload=211241]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.443119285s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................XX.....] duration: 3.373696759s FAIL (2025-08-17 04:25:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.373794197s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................................................XXXX....] duration: 2.71373115s FAIL (2025-08-17 04:25:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277217340 (202h56m58.236314588s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277444018 (202h56m58.236541264s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277693944 (202h56m58.236791191s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.277957287 (202h56m58.237054531s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278192843 (202h56m58.237290088s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278363507 (202h56m58.237460782s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.278887870 (202h56m58.237985121s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279265735 (202h56m58.238362991s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279639846 (202h56m58.238737091s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.279971414 (202h56m58.239068662s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280041108 (202h56m58.239138356s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280352792 (202h56m58.239450046s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280683314 (202h56m58.239780567s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.280969649 (202h56m58.240066896s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281241193 (202h56m58.240338438s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.281508639 (202h56m58.240605887s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Uploading piece... Aug 17 04:22:36.837376257 (202h56m58.796473489s) [upload=211240]: Next returned nil segment Aug 17 04:22:36.837399552 (202h56m58.796496798s) [upload=211240]: Waiting for error group managing segments... Aug 17 04:22:36.989047654 (202h56m58.948144887s) [noise=true , address=77.90.31.16:4761 , piece_id=BBQ6MRRN7VFPW4TERHX4OBGS5TABKCBX7E35MMXKNTROK2HCFYWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991378317 (202h56m58.950475552s) [noise=true , address=45.91.116.11:4125 , piece_id=7PHLM54VFQQWBFHCUCHXETPAJWQEKVV364UA2X26XMJGNTNL7AWQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:36.991998552 (202h56m58.951095785s) [noise=true , address=77.90.31.18:4823 , piece_id=X45AK2ZYZ5CO3EFPMS26ZACAVKQ7G6G22VG2QLI6XPB6P5IR7WYQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.010955158 (202h56m58.970052395s) [noise=true , address=77.90.31.19:4869 , piece_id=3QEFUP2SUEHZLB2PTCKJTLTEAOUZ3U4GU4K2NY5W7TIXCGVCWGVA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.017235751 (202h56m58.97633299s) [noise=true , address=77.90.31.21:4903 , piece_id=B2O6UPVOQEHVLGUXLAXAOE2GCGB4CALQZIYSWBHEYNEVHQWNAK3A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.045617264 (202h56m59.004714496s) [noise=true , address=77.90.31.10:4538 , piece_id=TXOA5355CRWXU2L2GL2EABPO2L3DR5BAQFUZVFQCZVQRBEZP66OA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.080535376 (202h56m59.039632621s) [noise=true , address=77.90.31.15:4701 , piece_id=IPGE5OY27GVJD65STT7JXTQCN7HSEOQAKNWLF4BIL66SCMIMFIHQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.121242355 (202h56m59.080339591s) [noise=true , address=45.91.116.12:4230 , piece_id=5Z44YMK6XO2UGZOH55FQDOMMAV3QPZH2XMW2RJHWSD6YEOYPM72Q, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.296208684 (202h56m59.255305934s) [noise=true , address=156.146.43.225:4453 , piece_id=D43T2X6NT2R5CU4ACAJJXKI7XMDXEV7GWV4KQ5TZCDO6BOC3JPIA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:22:37.528205336 (202h56m59.487302581s) [noise=true , address=66.23.193.175:4557 , piece_id=TCM2G7Z7J6R7SJAJWSU7PIJEPRQB6RYRTYMIXJVPFMYSHVPAH3IA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: <nil> Aug 17 04:23:35.761418960 (202h57m57.720516213s) [noise=true , address=104.36.82.60:4341 , piece_id=MKJHIPSBIZK7U5BL225UJVURW6L3QK4NFXLGZFJOH2LP3TM6U2SA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1okrDyM4TfzYN3p8zUjTqbw2661rKQQqZdP1ptAKzzTQ82C7cN, address:104.36.82.60:4341): protocol: rpc: dial tcp 104.36.82.60:4341: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761509741 (202h57m57.720606987s) [noise=true , address=104.36.82.34:4556 , piece_id=M4IYC36BRDPKHAQL4BRBM7U7IN4YQLRG5RRXVFXZ535NO5AGAQMA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1i62GRFASf7UPVGhUofmRkz3zZSyhq4fDycgyYtfD7pHR3w91v, address:104.36.82.34:4556): protocol: write tcp 139.59.144.22:37150->104.36.82.34:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761574924 (202h57m57.720672174s) [noise=true , address=104.36.82.17:4333 , piece_id=CGHZEP2ZKWES3L6XH4LSFLOFKYLQPYJZ56G36TSWA2MYHJORAL5A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1xRdTZoEk3NsnS445GcdEvwAdDzNLwUYSfgjB7bBSGRx9L9mrU, address:104.36.82.17:4333): protocol: write tcp 139.59.144.22:47980->104.36.82.17:4333: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761620359 (202h57m57.720717605s) [noise=true , address=104.36.82.58:4293 , piece_id=IY3EUM3SZ42OZA42BC2FSVTLTHU5POSMOOQT6MTLHNPZ6FEYE2EQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:15i783VxxzbriD7LZCbFWNBY8uCKUffXtmWE2eu2VupgE1t9bZ, address:104.36.82.58:4293): protocol: write tcp 139.59.144.22:39172->104.36.82.58:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761661063 (202h57m57.720758308s) [noise=true , address=104.36.82.36:4630 , piece_id=52F2JKA67HIID76G4BPIWQ7FIRZIDYLB5R6KXA2NZC45ULKY4ZXA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:12aETKgTzKw8fyYqbeDA2JzfxZnWn12xUPYK1eRt6TvBPPdRJDp, address:104.36.82.36:4630): protocol: write tcp 139.59.144.22:51970->104.36.82.36:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761681516 (202h57m57.720778766s) [noise=true , address=104.36.82.23:4474 , piece_id=2TJQ3BUKL5BTVHPAOEPKGQIZS4IX54HPYKIRLRAXLUTLMKXST6RA, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 139.59.144.22:35702->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761728458 (202h57m57.720825693s) [noise=true , address=104.36.82.9:4025 , piece_id=EJABFHKRXGUJKB565A56GVUO3I2ERZZEY4T2PZJ73V6MYF23GL2A, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:48846->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761769868 (202h57m57.720867115s) [noise=true , address=104.36.82.8:4992 , piece_id=NZP6GIHTMJJRNUHGZA4JPKFDEJ6LXESXVP33ZFLGR4QLDPVC7MIQ, seg_pos={0 0}, upload=211240]: Done uploading piece. err: ecclient: upload failed (node:1hbt8HKTunpPKyAzH4XBwGQqTKzcgqPpoWwKZm9JMtaK8P7aVp, address:104.36.82.8:4992): protocol: write tcp 139.59.144.22:40976->104.36.82.8:4992: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:35.761795616 (202h57m57.720892851s) [seg_pos={0 0}, upload=211240]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:35.867721911 (202h57m57.826819144s) [upload=211240]: Done uploading segments... Aug 17 04:23:35.867749737 (202h57m57.826846969s) [upload=211240]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 2.7137874s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....................................................XXXXX..] duration: 2.859245826s FAIL (2025-08-17 04:25:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.859386079s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......................................................XX....] duration: 3.583895414s FAIL (2025-08-17 04:25:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.583923965s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................................XXX.....] duration: 2.604430696s FAIL (2025-08-17 04:25:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 17 04:21:58.824378996 (202h56m20.783476241s) [upload=211227]: Starting upload Aug 17 04:21:58.824453728 (202h56m20.783550973s) [upload=211227]: Done starting upload Aug 17 04:21:58.824533712 (202h56m20.783630943s) [upload=211227]: Uploading segments... Aug 17 04:21:58.824814622 (202h56m20.783911863s) [upload=211227]: Got next segment. Inline: false Aug 17 04:21:58.824870724 (202h56m20.78396797s) [upload=211227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:21:58.824876461 (202h56m20.783973706s) [upload=211227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:21:59.025160521 (202h56m20.984257797s) [seg_pos={0 0}, upload=211227]: Begin upload segment... Aug 17 04:21:59.025238019 (202h56m20.984335266s) [seg_pos={0 0}, upload=211227]: Done begin upload segment. Aug 17 04:21:59.025273686 (202h56m20.98437092s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025331026 (202h56m20.984428309s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025756251 (202h56m20.984853496s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.025907425 (202h56m20.98500468s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026127117 (202h56m20.98522437s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026350346 (202h56m20.985447594s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026436670 (202h56m20.985533916s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026714369 (202h56m20.985811614s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.026788896 (202h56m20.985886144s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027068637 (202h56m20.986165882s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027120600 (202h56m20.986217846s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027420110 (202h56m20.986517355s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027425372 (202h56m20.98652262s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.027839122 (202h56m20.986936376s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.028247142 (202h56m20.987344396s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030005732 (202h56m20.989102986s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.030432900 (202h56m20.989530147s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.031846491 (202h56m20.990943746s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Uploading piece... Aug 17 04:21:59.588920323 (202h56m21.548017569s) [upload=211227]: Next returned nil segment Aug 17 04:21:59.588944741 (202h56m21.548041972s) [upload=211227]: Waiting for error group managing segments... Aug 17 04:21:59.741262552 (202h56m21.700359791s) [noise=true , address=45.91.116.12:4256 , piece_id=OITDESJQZQDVMAEJBQU4KAXWBY43JKXR7AHSE7OQ2TQKG5UKMQPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.742768290 (202h56m21.701865528s) [noise=true , address=77.90.31.11:4580 , piece_id=TS52Z2GSFQMC722KXJCVYXNMW7UT73PGVWMQFBAXSJZE77CTP7UA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.777622259 (202h56m21.736719503s) [noise=true , address=77.90.31.14:4673 , piece_id=GANKYC2PWGNMZXLY7ZTB5RUFQLTFDZW3CH4AXQJ4FYD673I4WTWQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.808692811 (202h56m21.767790057s) [noise=true , address=77.90.31.16:4745 , piece_id=SSPWQSSSZNDOSAVYSFO3WKTOH3MZT5WRW5S3ACYWYQGX54R7BDKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.831688332 (202h56m21.790785579s) [noise=true , address=77.90.31.19:4844 , piece_id=JKFXI6B5LZIIXLMF2UN3N226X3YX5ZRWPDJI6DYGRVZRAEOZEEIA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.871745680 (202h56m21.830842918s) [noise=true , address=77.90.31.10:4524 , piece_id=7KW7HUNUBCJWMG66YHQJ63H2UERBGQTIUV24U4WARUQWMHXEFSQQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.883314367 (202h56m21.84241162s) [noise=true , address=77.90.31.13:4641 , piece_id=SA6Z2Y3GPMFS2KSQNG2OX5QTEQ6UQMCSI7SAY2BR63NNOS37OTAA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.890311438 (202h56m21.849408688s) [noise=true , address=79.127.220.101:4068 , piece_id=WHJJIXY6ZBZJNQQKJEML5XP5DMRJGUTDRFMOWWD6DJNDSUWCEI6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.940441979 (202h56m21.899539228s) [noise=true , address=45.91.116.11:4132 , piece_id=YPR6JXF4CAZBHP5XZUA5QIK2RFG75HHIRYSQ33EDTNSRT545YY7Q, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:21:59.976656973 (202h56m21.935754218s) [noise=true , address=79.127.201.215:4357 , piece_id=5GN73I7DFN6XLU3D7VWZ7TTLYN2PQ2OAUVRRN4YOCZUUWL347EPQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.283185203 (202h56m22.242282448s) [noise=true , address=66.23.193.175:4553 , piece_id=DQWBH4FPAYOHWQKATN7XPI3B5RFBZ4MF7WUMVS6FR7L5CXNW3JCQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.284822985 (202h56m22.243920238s) [noise=true , address=64.49.13.26:4951 , piece_id=SJYX5R2QSB64W6M4H5XJCPGSNF5YMJYBRFJF7FPGSZQB4KBAFO6A, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:00.446988508 (202h56m22.406085753s) [noise=true , address=79.127.205.246:4275 , piece_id=ANNA5TM2H5ATR4ZCFZUW5Z2EFFKD6U2X2BWE7MB43RMPE4I7TADQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: <nil> Aug 17 04:22:58.545488378 (202h57m20.504585625s) [noise=true , address=104.36.82.5:4866 , piece_id=D54LIPVYHKYLIVZRYBOMNOLVYSK3EDH7WDEZ5E563AQ35FUBELAQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr, address:104.36.82.5:4866): protocol: write tcp 139.59.144.22:57798->104.36.82.5:4866: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545674721 (202h57m20.504771972s) [noise=true , address=104.36.82.23:4447 , piece_id=A56LTPVK6MNLOUCFJ7FXSAWFUOVQRLMUJB4PS5CYIEQTIWIG7DKQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 139.59.144.22:57190->104.36.82.23:4447: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545824338 (202h57m20.504921585s) [noise=true , address=104.36.82.44:4901 , piece_id=QYKMZVHQ3PNWZ2NPWR4QQNDJWFPZW3PKUQIP6BA2YAQA55R6JWDA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:1FHE84zvBb1MKwB1Ht7puTafgQxDEvPt4mYqkYVSSufAMpWQA1, address:104.36.82.44:4901): protocol: write tcp 139.59.144.22:54098->104.36.82.44:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545853626 (202h57m20.504950858s) [noise=true , address=104.36.82.35:4603 , piece_id=ZFG3KRVDGFTJPRYVOB7YFHAJDVS3FREBSOTHPZNHIAOOKNMAPJTA, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:12ASAeTMBu4sY1GuZiSuwTXmhPksA1Te2BVZdcoE9QvpLAmcQX3, address:104.36.82.35:4603): protocol: write tcp 139.59.144.22:49164->104.36.82.35:4603: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545907146 (202h57m20.505004393s) [noise=true , address=104.36.82.60:4365 , piece_id=4BYPCTHJK6JQKEDQFTQJD6CYZQFO2GGB4WSX7GWTRC4BT6QS5JEQ, seg_pos={0 0}, upload=211227]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: write tcp 139.59.144.22:43990->104.36.82.60:4365: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:58.545989950 (202h57m20.505087195s) [seg_pos={0 0}, upload=211227]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 04:22:58.646357515 (202h57m20.605454767s) [upload=211227]: Done uploading segments... Aug 17 04:22:58.646391150 (202h57m20.605488398s) [upload=211227]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.604470867s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................................................XXXX..] duration: 2.369176366s FAIL (2025-08-17 04:24:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Aug 17 04:22:36.193175039 (202h56m58.152272297s) [upload=211241]: Starting upload Aug 17 04:22:36.193282733 (202h56m58.152379987s) [upload=211241]: Done starting upload Aug 17 04:22:36.193533603 (202h56m58.152630849s) [upload=211241]: Uploading segments... Aug 17 04:22:36.193824638 (202h56m58.152921893s) [upload=211241]: Got next segment. Inline: false Aug 17 04:22:36.193891646 (202h56m58.152988897s) [upload=211241]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:22:36.193898487 (202h56m58.15299574s) [upload=211241]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:22:36.387313731 (202h56m58.346410984s) [seg_pos={0 0}, upload=211241]: Begin upload segment... Aug 17 04:22:36.387391508 (202h56m58.346488754s) [seg_pos={0 0}, upload=211241]: Done begin upload segment. Aug 17 04:22:36.387453494 (202h56m58.34655085s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.387549035 (202h56m58.346646289s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388292176 (202h56m58.347389421s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388564570 (202h56m58.347661824s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388775483 (202h56m58.347872728s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.388976887 (202h56m58.348074132s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389048807 (202h56m58.348146055s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389269495 (202h56m58.348366743s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389303339 (202h56m58.348400583s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389577693 (202h56m58.348674943s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389582267 (202h56m58.348679515s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389868029 (202h56m58.348965273s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.389899606 (202h56m58.348996853s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390181444 (202h56m58.349278692s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.390456412 (202h56m58.349553657s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:36.396850594 (202h56m58.355947842s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Uploading piece... Aug 17 04:22:37.003486038 (202h56m58.962583287s) [upload=211241]: Next returned nil segment Aug 17 04:22:37.003509888 (202h56m58.962607148s) [upload=211241]: Waiting for error group managing segments... Aug 17 04:22:37.141087302 (202h56m59.100184557s) [noise=true , address=45.91.116.4:4939 , piece_id=INXRARFNJD2KCP6IDBOCVAZA5U4QJX5TVPQALFXJZ5V6YNOGG5FQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.147724589 (202h56m59.106821827s) [noise=true , address=45.91.116.14:4190 , piece_id=RMPPQYC3JOXNR25LRTNKUHHVXSLWRYS4UK3WFL7JUN2IEB7MTPFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148191395 (202h56m59.107288641s) [noise=true , address=45.91.116.6:4974 , piece_id=SG5PHVVY6ZRIJLCQX6PW4BGKNNETDF3TWGM74GDYNJQUXPCG5DOA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.148249179 (202h56m59.107346437s) [noise=true , address=45.91.116.13:4153 , piece_id=WDOI5HF6IFTRHNWSZS7OCRVHJFWIKLOIUFKIO6HIA6DDD65V5AFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.154197920 (202h56m59.113295159s) [noise=true , address=45.91.116.11:4119 , piece_id=RTDAZ627L753OYJZNDWQEXNQKWNEOO4O6NXKVCAQJOFDR5O22AJQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155088755 (202h56m59.11418599s) [noise=true , address=45.91.116.9:4261 , piece_id=5WVHJ2XXGMLB5EFKDSI6XJD27MKZ3H76DR7M6NE4D4PUCNHZHU5A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.155737855 (202h56m59.114835102s) [noise=true , address=45.91.116.12:4226 , piece_id=FX6QBTY2YZK4RWYQQJICS23MPPGMPQY6CX3AEXKDJ26UQ35AIDVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.158542805 (202h56m59.117640044s) [noise=true , address=77.90.31.18:4807 , piece_id=PWOXWGYMFXZV55YHFVBSUTHPSUF2K5FEJADLMFWTICIRBXJGNW2A, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.163837059 (202h56m59.122934309s) [noise=true , address=45.91.116.5:4297 , piece_id=BKI5XCEXSAE4545PWGGL3RFHM5M5ZGBCAKLJYX2JFIUGJBC3LKXA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.225669823 (202h56m59.184767059s) [noise=true , address=45.91.116.7:4009 , piece_id=BQNCWPP6OVPYYTAMJPJCWACNVYW6BCSFDUH5GY7LRSYWHILWPULQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:22:37.249261694 (202h56m59.20835894s) [noise=true , address=77.90.31.14:4663 , piece_id=QHGIW4YTHWRUZC2GAJVRMTHMSTBMZTFSIWDXCQ72U5PHTPE6HDFA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: <nil> Aug 17 04:23:05.920185881 (202h57m27.879283118s) [noise=true , address=104.36.82.10:4079 , piece_id=JCA4FKNW3R7YPGU2FGO7OXK3JPI4J2NCSEYJNLXLX6EW6B4LNNSQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: rpc: dial tcp 104.36.82.10:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920277695 (202h57m27.879374941s) [noise=true , address=104.36.82.15:4259 , piece_id=L5C3UKGCVNKD52DSBG7OT53ON5NNG27WARZXUVK4DGRI5X2YITVA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: rpc: dial tcp 104.36.82.15:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920476811 (202h57m27.87957406s) [noise=true , address=104.36.82.7:4911 , piece_id=OL5RYLAFZM7YOJQXANQU5LIVSU7IFGN6WKB2N2TCRXHQGOILZ7JQ, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: rpc: dial tcp 104.36.82.7:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920550680 (202h57m27.879647915s) [noise=true , address=104.36.82.4:4801 , piece_id=RBMJ6IZSFMGIOZN3256PTIGBTYIQFYQFZFPHPFOZXNRTB5W3Z6HA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:05.920687798 (202h57m27.879785037s) [noise=true , address=104.36.82.8:4971 , piece_id=UNRIC3GXTSOWM266PVJYN3M33P474DKTAFTRQGJYYYV2VG2UH3YA, seg_pos={0 0}, upload=211241]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:52446->104.36.82.8:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 04:23:05.920736972 (202h57m27.879834217s) [seg_pos={0 0}, upload=211241]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Aug 17 04:23:06.024578477 (202h57m27.983675715s) [upload=211241]: Done uploading segments... Aug 17 04:23:06.024615107 (202h57m27.983712354s) [upload=211241]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.369207715s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)