Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X.....XX.............................................] duration: 2.573457834s FAIL (2025-08-17 13:55:38 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 2.573498809s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................................X.......X.................] duration: 963.177296ms FAIL (2025-08-17 13:55:28 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:42:51.584264741 (212h17m13.543361997s) [upload=221021]: Starting upload Aug 17 13:42:51.584345151 (212h17m13.543442383s) [upload=221021]: Done starting upload Aug 17 13:42:51.584440425 (212h17m13.543537656s) [upload=221021]: Uploading segments... Aug 17 13:42:51.584508717 (212h17m13.543605973s) [upload=221021]: Got next segment. Inline: true Aug 17 13:42:51.584515874 (212h17m13.543613131s) [upload=221021]: Waiting for error group managing segments... Aug 17 13:42:51.584584133 (212h17m13.543681388s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:42:51.584590380 (212h17m13.543687637s) [upload=221021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:42:51.584594159 (212h17m13.543691419s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:42:51.784353099 (212h17m13.743450345s) [upload=221021]: Done uploading segments... Aug 17 13:42:51.784373595 (212h17m13.743470834s) [upload=221021]: Upload finished. err: <nil> Duration:* 963.207361ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................................X.......X................] duration: 986.675791ms FAIL (2025-08-17 13:54:50 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:42:51.584264741 (212h17m13.543361997s) [upload=221021]: Starting upload Aug 17 13:42:51.584345151 (212h17m13.543442383s) [upload=221021]: Done starting upload Aug 17 13:42:51.584440425 (212h17m13.543537656s) [upload=221021]: Uploading segments... Aug 17 13:42:51.584508717 (212h17m13.543605973s) [upload=221021]: Got next segment. Inline: true Aug 17 13:42:51.584515874 (212h17m13.543613131s) [upload=221021]: Waiting for error group managing segments... Aug 17 13:42:51.584584133 (212h17m13.543681388s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:42:51.584590380 (212h17m13.543687637s) [upload=221021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:42:51.584594159 (212h17m13.543691419s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:42:51.784353099 (212h17m13.743450345s) [upload=221021]: Done uploading segments... Aug 17 13:42:51.784373595 (212h17m13.743470834s) [upload=221021]: Upload finished. err: <nil> Duration:* 986.696571ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X.....XX............................................] duration: 3.15333601s FAIL (2025-08-17 13:54:31 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 3.153379516s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....................................X.......X...............] duration: 993.506042ms FAIL (2025-08-17 13:54:17 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:42:51.584264741 (212h17m13.543361997s) [upload=221021]: Starting upload Aug 17 13:42:51.584345151 (212h17m13.543442383s) [upload=221021]: Done starting upload Aug 17 13:42:51.584440425 (212h17m13.543537656s) [upload=221021]: Uploading segments... Aug 17 13:42:51.584508717 (212h17m13.543605973s) [upload=221021]: Got next segment. Inline: true Aug 17 13:42:51.584515874 (212h17m13.543613131s) [upload=221021]: Waiting for error group managing segments... Aug 17 13:42:51.584584133 (212h17m13.543681388s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:42:51.584590380 (212h17m13.543687637s) [upload=221021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:42:51.584594159 (212h17m13.543691419s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:42:51.784353099 (212h17m13.743450345s) [upload=221021]: Done uploading segments... Aug 17 13:42:51.784373595 (212h17m13.743470834s) [upload=221021]: Upload finished. err: <nil> Duration:* 993.539236ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X.....XX...........................................] duration: 2.863139354s FAIL (2025-08-17 13:53:44 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 2.863189056s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X.......X..............] duration: 1.12182758s FAIL (2025-08-17 13:53:15 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:42:51.584264741 (212h17m13.543361997s) [upload=221021]: Starting upload Aug 17 13:42:51.584345151 (212h17m13.543442383s) [upload=221021]: Done starting upload Aug 17 13:42:51.584440425 (212h17m13.543537656s) [upload=221021]: Uploading segments... Aug 17 13:42:51.584508717 (212h17m13.543605973s) [upload=221021]: Got next segment. Inline: true Aug 17 13:42:51.584515874 (212h17m13.543613131s) [upload=221021]: Waiting for error group managing segments... Aug 17 13:42:51.584584133 (212h17m13.543681388s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:42:51.584590380 (212h17m13.543687637s) [upload=221021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:42:51.584594159 (212h17m13.543691419s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:42:51.784353099 (212h17m13.743450345s) [upload=221021]: Done uploading segments... Aug 17 13:42:51.784373595 (212h17m13.743470834s) [upload=221021]: Upload finished. err: <nil> Duration:* 1.12185879s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [X.........X.....XX..........................................] duration: 2.987481995s FAIL (2025-08-17 13:52:47 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 2.987512545s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......................................X.......X.............] duration: 1.074954041s FAIL (2025-08-17 13:52:21 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:42:51.584264741 (212h17m13.543361997s) [upload=221021]: Starting upload Aug 17 13:42:51.584345151 (212h17m13.543442383s) [upload=221021]: Done starting upload Aug 17 13:42:51.584440425 (212h17m13.543537656s) [upload=221021]: Uploading segments... Aug 17 13:42:51.584508717 (212h17m13.543605973s) [upload=221021]: Got next segment. Inline: true Aug 17 13:42:51.584515874 (212h17m13.543613131s) [upload=221021]: Waiting for error group managing segments... Aug 17 13:42:51.584584133 (212h17m13.543681388s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:42:51.584590380 (212h17m13.543687637s) [upload=221021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:42:51.584594159 (212h17m13.543691419s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:42:51.784353099 (212h17m13.743450345s) [upload=221021]: Done uploading segments... Aug 17 13:42:51.784373595 (212h17m13.743470834s) [upload=221021]: Upload finished. err: <nil> Duration:* 1.074985406s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X.........X.....XX.........................................] duration: 2.805808515s FAIL (2025-08-17 13:52:08 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 2.80584199s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X.......X............] duration: 931.839735ms FAIL (2025-08-17 13:51:59 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:42:51.584264741 (212h17m13.543361997s) [upload=221021]: Starting upload Aug 17 13:42:51.584345151 (212h17m13.543442383s) [upload=221021]: Done starting upload Aug 17 13:42:51.584440425 (212h17m13.543537656s) [upload=221021]: Uploading segments... Aug 17 13:42:51.584508717 (212h17m13.543605973s) [upload=221021]: Got next segment. Inline: true Aug 17 13:42:51.584515874 (212h17m13.543613131s) [upload=221021]: Waiting for error group managing segments... Aug 17 13:42:51.584584133 (212h17m13.543681388s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:42:51.584590380 (212h17m13.543687637s) [upload=221021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:42:51.584594159 (212h17m13.543691419s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:42:51.784353099 (212h17m13.743450345s) [upload=221021]: Done uploading segments... Aug 17 13:42:51.784373595 (212h17m13.743470834s) [upload=221021]: Upload finished. err: <nil> Duration:* 931.883936ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [X.X.........X.....XX........................................] duration: 3.275369855s FAIL (2025-08-17 13:51:25 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 3.27542755s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................................X.......X...........] duration: 880.542507ms FAIL (2025-08-17 13:51:23 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:42:51.584264741 (212h17m13.543361997s) [upload=221021]: Starting upload Aug 17 13:42:51.584345151 (212h17m13.543442383s) [upload=221021]: Done starting upload Aug 17 13:42:51.584440425 (212h17m13.543537656s) [upload=221021]: Uploading segments... Aug 17 13:42:51.584508717 (212h17m13.543605973s) [upload=221021]: Got next segment. Inline: true Aug 17 13:42:51.584515874 (212h17m13.543613131s) [upload=221021]: Waiting for error group managing segments... Aug 17 13:42:51.584584133 (212h17m13.543681388s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:42:51.584590380 (212h17m13.543687637s) [upload=221021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:42:51.584594159 (212h17m13.543691419s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:42:51.784353099 (212h17m13.743450345s) [upload=221021]: Done uploading segments... Aug 17 13:42:51.784373595 (212h17m13.743470834s) [upload=221021]: Upload finished. err: <nil> Duration:* 880.572791ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.30370268s FAIL (2025-08-17 13:50:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 13:04:48.671168571 (211h39m10.630265822s) [upload=220354]: Starting upload Aug 17 13:04:48.671289154 (211h39m10.630386405s) [upload=220354]: Done starting upload Aug 17 13:04:48.671375030 (211h39m10.630472267s) [upload=220354]: Uploading segments... Aug 17 13:04:48.671685291 (211h39m10.630782528s) [upload=220354]: Got next segment. Inline: false Aug 17 13:04:48.671859547 (211h39m10.630956792s) [upload=220354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:48.671868198 (211h39m10.630965451s) [upload=220354]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:48.784496066 (211h39m10.743593311s) [seg_pos={0 0}, upload=220354]: Begin upload segment... Aug 17 13:04:48.784546278 (211h39m10.743643523s) [seg_pos={0 0}, upload=220354]: Done begin upload segment. Aug 17 13:04:48.784576969 (211h39m10.743674214s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784591947 (211h39m10.743689244s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784926858 (211h39m10.744024104s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785024612 (211h39m10.744121858s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785177382 (211h39m10.744274627s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785386459 (211h39m10.744483708s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785426090 (211h39m10.744523335s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785675714 (211h39m10.744772959s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786030292 (211h39m10.745127541s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786370888 (211h39m10.745468139s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786497361 (211h39m10.745594613s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786709369 (211h39m10.745806618s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787016861 (211h39m10.746114097s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787272873 (211h39m10.746370129s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787671377 (211h39m10.746768629s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.788114039 (211h39m10.747211295s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:49.396438949 (211h39m11.355536206s) [upload=220354]: Next returned nil segment Aug 17 13:04:49.396459727 (211h39m11.355556984s) [upload=220354]: Waiting for error group managing segments... Aug 17 13:04:49.548390642 (211h39m11.507487888s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.555382402 (211h39m11.514479641s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.568461124 (211h39m11.527558355s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.594216288 (211h39m11.553313519s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.596651299 (211h39m11.555748557s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.600004059 (211h39m11.559101306s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.619246354 (211h39m11.578343598s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.622590623 (211h39m11.581687861s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.633199644 (211h39m11.592296882s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.638567248 (211h39m11.597664486s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:05:18.402418747 (211h39m40.361515993s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: rpc: dial tcp 104.36.82.13:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402498626 (211h39m40.361595873s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402587513 (211h39m40.361684745s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402653738 (211h39m40.361750984s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402768644 (211h39m40.361865891s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402931362 (211h39m40.362028611s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402953112 (211h39m40.362050358s) [seg_pos={0 0}, upload=220354]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 13:05:18.518203938 (211h39m40.477301184s) [upload=220354]: Done uploading segments... Aug 17 13:05:18.518239058 (211h39m40.477336313s) [upload=220354]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.303727292s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X.......X..........] duration: 877.518358ms FAIL (2025-08-17 13:50:37 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:42:51.584264741 (212h17m13.543361997s) [upload=221021]: Starting upload Aug 17 13:42:51.584345151 (212h17m13.543442383s) [upload=221021]: Done starting upload Aug 17 13:42:51.584440425 (212h17m13.543537656s) [upload=221021]: Uploading segments... Aug 17 13:42:51.584508717 (212h17m13.543605973s) [upload=221021]: Got next segment. Inline: true Aug 17 13:42:51.584515874 (212h17m13.543613131s) [upload=221021]: Waiting for error group managing segments... Aug 17 13:42:51.584584133 (212h17m13.543681388s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:42:51.584590380 (212h17m13.543687637s) [upload=221021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:42:51.584594159 (212h17m13.543691419s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:42:51.784353099 (212h17m13.743450345s) [upload=221021]: Done uploading segments... Aug 17 13:42:51.784373595 (212h17m13.743470834s) [upload=221021]: Upload finished. err: <nil> Duration:* 877.567449ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [XX.X.........X.....XX.......................................] duration: 3.073194978s FAIL (2025-08-17 13:50:31 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 3.073231616s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 2.559259364s FAIL (2025-08-17 13:50:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 13:04:48.671168571 (211h39m10.630265822s) [upload=220354]: Starting upload Aug 17 13:04:48.671289154 (211h39m10.630386405s) [upload=220354]: Done starting upload Aug 17 13:04:48.671375030 (211h39m10.630472267s) [upload=220354]: Uploading segments... Aug 17 13:04:48.671685291 (211h39m10.630782528s) [upload=220354]: Got next segment. Inline: false Aug 17 13:04:48.671859547 (211h39m10.630956792s) [upload=220354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:48.671868198 (211h39m10.630965451s) [upload=220354]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:48.784496066 (211h39m10.743593311s) [seg_pos={0 0}, upload=220354]: Begin upload segment... Aug 17 13:04:48.784546278 (211h39m10.743643523s) [seg_pos={0 0}, upload=220354]: Done begin upload segment. Aug 17 13:04:48.784576969 (211h39m10.743674214s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784591947 (211h39m10.743689244s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784926858 (211h39m10.744024104s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785024612 (211h39m10.744121858s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785177382 (211h39m10.744274627s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785386459 (211h39m10.744483708s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785426090 (211h39m10.744523335s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785675714 (211h39m10.744772959s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786030292 (211h39m10.745127541s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786370888 (211h39m10.745468139s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786497361 (211h39m10.745594613s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786709369 (211h39m10.745806618s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787016861 (211h39m10.746114097s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787272873 (211h39m10.746370129s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787671377 (211h39m10.746768629s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.788114039 (211h39m10.747211295s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:49.396438949 (211h39m11.355536206s) [upload=220354]: Next returned nil segment Aug 17 13:04:49.396459727 (211h39m11.355556984s) [upload=220354]: Waiting for error group managing segments... Aug 17 13:04:49.548390642 (211h39m11.507487888s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.555382402 (211h39m11.514479641s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.568461124 (211h39m11.527558355s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.594216288 (211h39m11.553313519s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.596651299 (211h39m11.555748557s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.600004059 (211h39m11.559101306s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.619246354 (211h39m11.578343598s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.622590623 (211h39m11.581687861s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.633199644 (211h39m11.592296882s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.638567248 (211h39m11.597664486s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:05:18.402418747 (211h39m40.361515993s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: rpc: dial tcp 104.36.82.13:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402498626 (211h39m40.361595873s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402587513 (211h39m40.361684745s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402653738 (211h39m40.361750984s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402768644 (211h39m40.361865891s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402931362 (211h39m40.362028611s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402953112 (211h39m40.362050358s) [seg_pos={0 0}, upload=220354]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 13:05:18.518203938 (211h39m40.477301184s) [upload=220354]: Done uploading segments... Aug 17 13:05:18.518239058 (211h39m40.477336313s) [upload=220354]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.55938656s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [XXX.X.........X.....XX......................................] duration: 2.640179212s FAIL (2025-08-17 13:50:16 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 2.64021265s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X.......X.........] duration: 860.328575ms FAIL (2025-08-17 13:50:01 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:42:51.584264741 (212h17m13.543361997s) [upload=221021]: Starting upload Aug 17 13:42:51.584345151 (212h17m13.543442383s) [upload=221021]: Done starting upload Aug 17 13:42:51.584440425 (212h17m13.543537656s) [upload=221021]: Uploading segments... Aug 17 13:42:51.584508717 (212h17m13.543605973s) [upload=221021]: Got next segment. Inline: true Aug 17 13:42:51.584515874 (212h17m13.543613131s) [upload=221021]: Waiting for error group managing segments... Aug 17 13:42:51.584584133 (212h17m13.543681388s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:42:51.584590380 (212h17m13.543687637s) [upload=221021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:42:51.584594159 (212h17m13.543691419s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:42:51.784353099 (212h17m13.743450345s) [upload=221021]: Done uploading segments... Aug 17 13:42:51.784373595 (212h17m13.743470834s) [upload=221021]: Upload finished. err: <nil> Duration:* 860.362577ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [.XXX.X.........X.....XX.....................................] duration: 2.575888717s FAIL (2025-08-17 13:49:59 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 2.575924594s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XXX.........................................................] duration: 2.240336402s FAIL (2025-08-17 13:49:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 13:04:48.671168571 (211h39m10.630265822s) [upload=220354]: Starting upload Aug 17 13:04:48.671289154 (211h39m10.630386405s) [upload=220354]: Done starting upload Aug 17 13:04:48.671375030 (211h39m10.630472267s) [upload=220354]: Uploading segments... Aug 17 13:04:48.671685291 (211h39m10.630782528s) [upload=220354]: Got next segment. Inline: false Aug 17 13:04:48.671859547 (211h39m10.630956792s) [upload=220354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:48.671868198 (211h39m10.630965451s) [upload=220354]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:48.784496066 (211h39m10.743593311s) [seg_pos={0 0}, upload=220354]: Begin upload segment... Aug 17 13:04:48.784546278 (211h39m10.743643523s) [seg_pos={0 0}, upload=220354]: Done begin upload segment. Aug 17 13:04:48.784576969 (211h39m10.743674214s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784591947 (211h39m10.743689244s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784926858 (211h39m10.744024104s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785024612 (211h39m10.744121858s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785177382 (211h39m10.744274627s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785386459 (211h39m10.744483708s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785426090 (211h39m10.744523335s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785675714 (211h39m10.744772959s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786030292 (211h39m10.745127541s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786370888 (211h39m10.745468139s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786497361 (211h39m10.745594613s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786709369 (211h39m10.745806618s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787016861 (211h39m10.746114097s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787272873 (211h39m10.746370129s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787671377 (211h39m10.746768629s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.788114039 (211h39m10.747211295s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:49.396438949 (211h39m11.355536206s) [upload=220354]: Next returned nil segment Aug 17 13:04:49.396459727 (211h39m11.355556984s) [upload=220354]: Waiting for error group managing segments... Aug 17 13:04:49.548390642 (211h39m11.507487888s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.555382402 (211h39m11.514479641s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.568461124 (211h39m11.527558355s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.594216288 (211h39m11.553313519s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.596651299 (211h39m11.555748557s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.600004059 (211h39m11.559101306s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.619246354 (211h39m11.578343598s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.622590623 (211h39m11.581687861s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.633199644 (211h39m11.592296882s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.638567248 (211h39m11.597664486s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:05:18.402418747 (211h39m40.361515993s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: rpc: dial tcp 104.36.82.13:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402498626 (211h39m40.361595873s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402587513 (211h39m40.361684745s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402653738 (211h39m40.361750984s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402768644 (211h39m40.361865891s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402931362 (211h39m40.362028611s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402953112 (211h39m40.362050358s) [seg_pos={0 0}, upload=220354]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 13:05:18.518203938 (211h39m40.477301184s) [upload=220354]: Done uploading segments... Aug 17 13:05:18.518239058 (211h39m40.477336313s) [upload=220354]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.240366961s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [X..........................................X.......X........] duration: 940.669996ms FAIL (2025-08-17 13:49:08 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:42:51.584264741 (212h17m13.543361997s) [upload=221021]: Starting upload Aug 17 13:42:51.584345151 (212h17m13.543442383s) [upload=221021]: Done starting upload Aug 17 13:42:51.584440425 (212h17m13.543537656s) [upload=221021]: Uploading segments... Aug 17 13:42:51.584508717 (212h17m13.543605973s) [upload=221021]: Got next segment. Inline: true Aug 17 13:42:51.584515874 (212h17m13.543613131s) [upload=221021]: Waiting for error group managing segments... Aug 17 13:42:51.584584133 (212h17m13.543681388s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:42:51.584590380 (212h17m13.543687637s) [upload=221021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:42:51.584594159 (212h17m13.543691419s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:42:51.784353099 (212h17m13.743450345s) [upload=221021]: Done uploading segments... Aug 17 13:42:51.784373595 (212h17m13.743470834s) [upload=221021]: Upload finished. err: <nil> Duration:* 940.69588ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [..XXX.X.........X.....XX....................................] duration: 2.739319999s FAIL (2025-08-17 13:49:05 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 2.739353444s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.X..........................................X.......X.......] duration: 869.944795ms FAIL (2025-08-17 13:48:39 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:42:51.584264741 (212h17m13.543361997s) [upload=221021]: Starting upload Aug 17 13:42:51.584345151 (212h17m13.543442383s) [upload=221021]: Done starting upload Aug 17 13:42:51.584440425 (212h17m13.543537656s) [upload=221021]: Uploading segments... Aug 17 13:42:51.584508717 (212h17m13.543605973s) [upload=221021]: Got next segment. Inline: true Aug 17 13:42:51.584515874 (212h17m13.543613131s) [upload=221021]: Waiting for error group managing segments... Aug 17 13:42:51.584584133 (212h17m13.543681388s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:42:51.584590380 (212h17m13.543687637s) [upload=221021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:42:51.584594159 (212h17m13.543691419s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:42:51.784353099 (212h17m13.743450345s) [upload=221021]: Done uploading segments... Aug 17 13:42:51.784373595 (212h17m13.743470834s) [upload=221021]: Upload finished. err: <nil> Duration:* 869.978257ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [XXXX........................................................] duration: 2.478040786s FAIL (2025-08-17 13:48:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 13:04:48.671168571 (211h39m10.630265822s) [upload=220354]: Starting upload Aug 17 13:04:48.671289154 (211h39m10.630386405s) [upload=220354]: Done starting upload Aug 17 13:04:48.671375030 (211h39m10.630472267s) [upload=220354]: Uploading segments... Aug 17 13:04:48.671685291 (211h39m10.630782528s) [upload=220354]: Got next segment. Inline: false Aug 17 13:04:48.671859547 (211h39m10.630956792s) [upload=220354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:48.671868198 (211h39m10.630965451s) [upload=220354]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:48.784496066 (211h39m10.743593311s) [seg_pos={0 0}, upload=220354]: Begin upload segment... Aug 17 13:04:48.784546278 (211h39m10.743643523s) [seg_pos={0 0}, upload=220354]: Done begin upload segment. Aug 17 13:04:48.784576969 (211h39m10.743674214s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784591947 (211h39m10.743689244s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784926858 (211h39m10.744024104s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785024612 (211h39m10.744121858s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785177382 (211h39m10.744274627s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785386459 (211h39m10.744483708s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785426090 (211h39m10.744523335s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785675714 (211h39m10.744772959s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786030292 (211h39m10.745127541s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786370888 (211h39m10.745468139s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786497361 (211h39m10.745594613s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786709369 (211h39m10.745806618s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787016861 (211h39m10.746114097s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787272873 (211h39m10.746370129s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787671377 (211h39m10.746768629s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.788114039 (211h39m10.747211295s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:49.396438949 (211h39m11.355536206s) [upload=220354]: Next returned nil segment Aug 17 13:04:49.396459727 (211h39m11.355556984s) [upload=220354]: Waiting for error group managing segments... Aug 17 13:04:49.548390642 (211h39m11.507487888s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.555382402 (211h39m11.514479641s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.568461124 (211h39m11.527558355s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.594216288 (211h39m11.553313519s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.596651299 (211h39m11.555748557s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.600004059 (211h39m11.559101306s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.619246354 (211h39m11.578343598s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.622590623 (211h39m11.581687861s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.633199644 (211h39m11.592296882s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.638567248 (211h39m11.597664486s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:05:18.402418747 (211h39m40.361515993s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: rpc: dial tcp 104.36.82.13:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402498626 (211h39m40.361595873s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402587513 (211h39m40.361684745s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402653738 (211h39m40.361750984s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402768644 (211h39m40.361865891s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402931362 (211h39m40.362028611s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402953112 (211h39m40.362050358s) [seg_pos={0 0}, upload=220354]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 13:05:18.518203938 (211h39m40.477301184s) [upload=220354]: Done uploading segments... Aug 17 13:05:18.518239058 (211h39m40.477336313s) [upload=220354]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.478081592s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [...XXX.X.........X.....XX...................................] duration: 2.747163952s FAIL (2025-08-17 13:47:59 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 2.747220905s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.7394183s FAIL (2025-08-17 13:47:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.739449453s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 2.601831824s FAIL (2025-08-17 13:47:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.601861433s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..X..........................................X.......X......] duration: 860.309757ms FAIL (2025-08-17 13:47:25 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:42:51.584264741 (212h17m13.543361997s) [upload=221021]: Starting upload Aug 17 13:42:51.584345151 (212h17m13.543442383s) [upload=221021]: Done starting upload Aug 17 13:42:51.584440425 (212h17m13.543537656s) [upload=221021]: Uploading segments... Aug 17 13:42:51.584508717 (212h17m13.543605973s) [upload=221021]: Got next segment. Inline: true Aug 17 13:42:51.584515874 (212h17m13.543613131s) [upload=221021]: Waiting for error group managing segments... Aug 17 13:42:51.584584133 (212h17m13.543681388s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:42:51.584590380 (212h17m13.543687637s) [upload=221021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:42:51.584594159 (212h17m13.543691419s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:42:51.784353099 (212h17m13.743450345s) [upload=221021]: Done uploading segments... Aug 17 13:42:51.784373595 (212h17m13.743470834s) [upload=221021]: Upload finished. err: <nil> Duration:* 860.356528ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.XXXX.......................................................] duration: 2.563903309s FAIL (2025-08-17 13:47:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 13:04:48.671168571 (211h39m10.630265822s) [upload=220354]: Starting upload Aug 17 13:04:48.671289154 (211h39m10.630386405s) [upload=220354]: Done starting upload Aug 17 13:04:48.671375030 (211h39m10.630472267s) [upload=220354]: Uploading segments... Aug 17 13:04:48.671685291 (211h39m10.630782528s) [upload=220354]: Got next segment. Inline: false Aug 17 13:04:48.671859547 (211h39m10.630956792s) [upload=220354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:48.671868198 (211h39m10.630965451s) [upload=220354]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:48.784496066 (211h39m10.743593311s) [seg_pos={0 0}, upload=220354]: Begin upload segment... Aug 17 13:04:48.784546278 (211h39m10.743643523s) [seg_pos={0 0}, upload=220354]: Done begin upload segment. Aug 17 13:04:48.784576969 (211h39m10.743674214s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784591947 (211h39m10.743689244s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784926858 (211h39m10.744024104s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785024612 (211h39m10.744121858s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785177382 (211h39m10.744274627s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785386459 (211h39m10.744483708s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785426090 (211h39m10.744523335s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785675714 (211h39m10.744772959s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786030292 (211h39m10.745127541s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786370888 (211h39m10.745468139s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786497361 (211h39m10.745594613s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786709369 (211h39m10.745806618s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787016861 (211h39m10.746114097s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787272873 (211h39m10.746370129s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787671377 (211h39m10.746768629s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.788114039 (211h39m10.747211295s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:49.396438949 (211h39m11.355536206s) [upload=220354]: Next returned nil segment Aug 17 13:04:49.396459727 (211h39m11.355556984s) [upload=220354]: Waiting for error group managing segments... Aug 17 13:04:49.548390642 (211h39m11.507487888s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.555382402 (211h39m11.514479641s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.568461124 (211h39m11.527558355s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.594216288 (211h39m11.553313519s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.596651299 (211h39m11.555748557s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.600004059 (211h39m11.559101306s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.619246354 (211h39m11.578343598s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.622590623 (211h39m11.581687861s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.633199644 (211h39m11.592296882s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.638567248 (211h39m11.597664486s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:05:18.402418747 (211h39m40.361515993s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: rpc: dial tcp 104.36.82.13:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402498626 (211h39m40.361595873s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402587513 (211h39m40.361684745s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402653738 (211h39m40.361750984s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402768644 (211h39m40.361865891s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402931362 (211h39m40.362028611s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402953112 (211h39m40.362050358s) [seg_pos={0 0}, upload=220354]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 13:05:18.518203938 (211h39m40.477301184s) [upload=220354]: Done uploading segments... Aug 17 13:05:18.518239058 (211h39m40.477336313s) [upload=220354]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.563938896s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [....XXX.X.........X.....XX..................................] duration: 2.673819527s FAIL (2025-08-17 13:46:59 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 2.673878668s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....XXX.X.........X.....XX.................................] duration: 2.972846183s FAIL (2025-08-17 13:46:33 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 2.972900949s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...X..........................................X.......X.....] duration: 933.874089ms FAIL (2025-08-17 13:46:32 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:42:51.584264741 (212h17m13.543361997s) [upload=221021]: Starting upload Aug 17 13:42:51.584345151 (212h17m13.543442383s) [upload=221021]: Done starting upload Aug 17 13:42:51.584440425 (212h17m13.543537656s) [upload=221021]: Uploading segments... Aug 17 13:42:51.584508717 (212h17m13.543605973s) [upload=221021]: Got next segment. Inline: true Aug 17 13:42:51.584515874 (212h17m13.543613131s) [upload=221021]: Waiting for error group managing segments... Aug 17 13:42:51.584584133 (212h17m13.543681388s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:42:51.584590380 (212h17m13.543687637s) [upload=221021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:42:51.584594159 (212h17m13.543691419s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:42:51.784353099 (212h17m13.743450345s) [upload=221021]: Done uploading segments... Aug 17 13:42:51.784373595 (212h17m13.743470834s) [upload=221021]: Upload finished. err: <nil> Duration:* 933.908773ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [XXX.........................................................] duration: 2.811946599s FAIL (2025-08-17 13:46:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.81198707s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..XXXX......................................................] duration: 2.483851978s FAIL (2025-08-17 13:46:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 13:04:48.671168571 (211h39m10.630265822s) [upload=220354]: Starting upload Aug 17 13:04:48.671289154 (211h39m10.630386405s) [upload=220354]: Done starting upload Aug 17 13:04:48.671375030 (211h39m10.630472267s) [upload=220354]: Uploading segments... Aug 17 13:04:48.671685291 (211h39m10.630782528s) [upload=220354]: Got next segment. Inline: false Aug 17 13:04:48.671859547 (211h39m10.630956792s) [upload=220354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:48.671868198 (211h39m10.630965451s) [upload=220354]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:48.784496066 (211h39m10.743593311s) [seg_pos={0 0}, upload=220354]: Begin upload segment... Aug 17 13:04:48.784546278 (211h39m10.743643523s) [seg_pos={0 0}, upload=220354]: Done begin upload segment. Aug 17 13:04:48.784576969 (211h39m10.743674214s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784591947 (211h39m10.743689244s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784926858 (211h39m10.744024104s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785024612 (211h39m10.744121858s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785177382 (211h39m10.744274627s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785386459 (211h39m10.744483708s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785426090 (211h39m10.744523335s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785675714 (211h39m10.744772959s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786030292 (211h39m10.745127541s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786370888 (211h39m10.745468139s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786497361 (211h39m10.745594613s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786709369 (211h39m10.745806618s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787016861 (211h39m10.746114097s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787272873 (211h39m10.746370129s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787671377 (211h39m10.746768629s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.788114039 (211h39m10.747211295s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:49.396438949 (211h39m11.355536206s) [upload=220354]: Next returned nil segment Aug 17 13:04:49.396459727 (211h39m11.355556984s) [upload=220354]: Waiting for error group managing segments... Aug 17 13:04:49.548390642 (211h39m11.507487888s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.555382402 (211h39m11.514479641s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.568461124 (211h39m11.527558355s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.594216288 (211h39m11.553313519s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.596651299 (211h39m11.555748557s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.600004059 (211h39m11.559101306s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.619246354 (211h39m11.578343598s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.622590623 (211h39m11.581687861s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.633199644 (211h39m11.592296882s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.638567248 (211h39m11.597664486s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:05:18.402418747 (211h39m40.361515993s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: rpc: dial tcp 104.36.82.13:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402498626 (211h39m40.361595873s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402587513 (211h39m40.361684745s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402653738 (211h39m40.361750984s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402768644 (211h39m40.361865891s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402931362 (211h39m40.362028611s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402953112 (211h39m40.362050358s) [seg_pos={0 0}, upload=220354]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 13:05:18.518203938 (211h39m40.477301184s) [upload=220354]: Done uploading segments... Aug 17 13:05:18.518239058 (211h39m40.477336313s) [upload=220354]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.483887405s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [XXXX........................................................] duration: 3.068763188s FAIL (2025-08-17 13:45:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 3.068881342s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [....X..........................................X.......X....] duration: 911.857807ms FAIL (2025-08-17 13:45:24 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:42:51.584264741 (212h17m13.543361997s) [upload=221021]: Starting upload Aug 17 13:42:51.584345151 (212h17m13.543442383s) [upload=221021]: Done starting upload Aug 17 13:42:51.584440425 (212h17m13.543537656s) [upload=221021]: Uploading segments... Aug 17 13:42:51.584508717 (212h17m13.543605973s) [upload=221021]: Got next segment. Inline: true Aug 17 13:42:51.584515874 (212h17m13.543613131s) [upload=221021]: Waiting for error group managing segments... Aug 17 13:42:51.584584133 (212h17m13.543681388s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:42:51.584590380 (212h17m13.543687637s) [upload=221021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:42:51.584594159 (212h17m13.543691419s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:42:51.784353099 (212h17m13.743450345s) [upload=221021]: Done uploading segments... Aug 17 13:42:51.784373595 (212h17m13.743470834s) [upload=221021]: Upload finished. err: <nil> Duration:* 911.899594ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [......XXX.X.........X.....XX................................] duration: 2.663867076s FAIL (2025-08-17 13:45:18 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 2.663905885s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.XXXX.......................................................] duration: 2.788365717s FAIL (2025-08-17 13:45:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.788398174s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...XXXX.....................................................] duration: 2.342310113s FAIL (2025-08-17 13:45:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 13:04:48.671168571 (211h39m10.630265822s) [upload=220354]: Starting upload Aug 17 13:04:48.671289154 (211h39m10.630386405s) [upload=220354]: Done starting upload Aug 17 13:04:48.671375030 (211h39m10.630472267s) [upload=220354]: Uploading segments... Aug 17 13:04:48.671685291 (211h39m10.630782528s) [upload=220354]: Got next segment. Inline: false Aug 17 13:04:48.671859547 (211h39m10.630956792s) [upload=220354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:48.671868198 (211h39m10.630965451s) [upload=220354]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:48.784496066 (211h39m10.743593311s) [seg_pos={0 0}, upload=220354]: Begin upload segment... Aug 17 13:04:48.784546278 (211h39m10.743643523s) [seg_pos={0 0}, upload=220354]: Done begin upload segment. Aug 17 13:04:48.784576969 (211h39m10.743674214s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784591947 (211h39m10.743689244s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784926858 (211h39m10.744024104s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785024612 (211h39m10.744121858s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785177382 (211h39m10.744274627s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785386459 (211h39m10.744483708s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785426090 (211h39m10.744523335s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785675714 (211h39m10.744772959s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786030292 (211h39m10.745127541s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786370888 (211h39m10.745468139s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786497361 (211h39m10.745594613s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786709369 (211h39m10.745806618s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787016861 (211h39m10.746114097s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787272873 (211h39m10.746370129s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787671377 (211h39m10.746768629s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.788114039 (211h39m10.747211295s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:49.396438949 (211h39m11.355536206s) [upload=220354]: Next returned nil segment Aug 17 13:04:49.396459727 (211h39m11.355556984s) [upload=220354]: Waiting for error group managing segments... Aug 17 13:04:49.548390642 (211h39m11.507487888s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.555382402 (211h39m11.514479641s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.568461124 (211h39m11.527558355s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.594216288 (211h39m11.553313519s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.596651299 (211h39m11.555748557s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.600004059 (211h39m11.559101306s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.619246354 (211h39m11.578343598s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.622590623 (211h39m11.581687861s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.633199644 (211h39m11.592296882s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.638567248 (211h39m11.597664486s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:05:18.402418747 (211h39m40.361515993s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: rpc: dial tcp 104.36.82.13:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402498626 (211h39m40.361595873s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402587513 (211h39m40.361684745s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402653738 (211h39m40.361750984s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402768644 (211h39m40.361865891s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402931362 (211h39m40.362028611s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402953112 (211h39m40.362050358s) [seg_pos={0 0}, upload=220354]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 13:05:18.518203938 (211h39m40.477301184s) [upload=220354]: Done uploading segments... Aug 17 13:05:18.518239058 (211h39m40.477336313s) [upload=220354]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.342349701s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.....X..........................................X.......X...] duration: 848.814755ms FAIL (2025-08-17 13:44:54 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:42:51.584264741 (212h17m13.543361997s) [upload=221021]: Starting upload Aug 17 13:42:51.584345151 (212h17m13.543442383s) [upload=221021]: Done starting upload Aug 17 13:42:51.584440425 (212h17m13.543537656s) [upload=221021]: Uploading segments... Aug 17 13:42:51.584508717 (212h17m13.543605973s) [upload=221021]: Got next segment. Inline: true Aug 17 13:42:51.584515874 (212h17m13.543613131s) [upload=221021]: Waiting for error group managing segments... Aug 17 13:42:51.584584133 (212h17m13.543681388s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:42:51.584590380 (212h17m13.543687637s) [upload=221021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:42:51.584594159 (212h17m13.543691419s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:42:51.784353099 (212h17m13.743450345s) [upload=221021]: Done uploading segments... Aug 17 13:42:51.784373595 (212h17m13.743470834s) [upload=221021]: Upload finished. err: <nil> Duration:* 848.864096ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..XXXX......................................................] duration: 2.756355212s FAIL (2025-08-17 13:44:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.756387359s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [.......XXX.X.........X.....XX...............................] duration: 2.835100053s FAIL (2025-08-17 13:44:25 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 2.835135267s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....XXXX....................................................] duration: 2.308132335s FAIL (2025-08-17 13:44:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 13:04:48.671168571 (211h39m10.630265822s) [upload=220354]: Starting upload Aug 17 13:04:48.671289154 (211h39m10.630386405s) [upload=220354]: Done starting upload Aug 17 13:04:48.671375030 (211h39m10.630472267s) [upload=220354]: Uploading segments... Aug 17 13:04:48.671685291 (211h39m10.630782528s) [upload=220354]: Got next segment. Inline: false Aug 17 13:04:48.671859547 (211h39m10.630956792s) [upload=220354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:48.671868198 (211h39m10.630965451s) [upload=220354]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:48.784496066 (211h39m10.743593311s) [seg_pos={0 0}, upload=220354]: Begin upload segment... Aug 17 13:04:48.784546278 (211h39m10.743643523s) [seg_pos={0 0}, upload=220354]: Done begin upload segment. Aug 17 13:04:48.784576969 (211h39m10.743674214s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784591947 (211h39m10.743689244s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784926858 (211h39m10.744024104s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785024612 (211h39m10.744121858s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785177382 (211h39m10.744274627s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785386459 (211h39m10.744483708s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785426090 (211h39m10.744523335s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785675714 (211h39m10.744772959s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786030292 (211h39m10.745127541s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786370888 (211h39m10.745468139s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786497361 (211h39m10.745594613s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786709369 (211h39m10.745806618s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787016861 (211h39m10.746114097s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787272873 (211h39m10.746370129s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787671377 (211h39m10.746768629s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.788114039 (211h39m10.747211295s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:49.396438949 (211h39m11.355536206s) [upload=220354]: Next returned nil segment Aug 17 13:04:49.396459727 (211h39m11.355556984s) [upload=220354]: Waiting for error group managing segments... Aug 17 13:04:49.548390642 (211h39m11.507487888s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.555382402 (211h39m11.514479641s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.568461124 (211h39m11.527558355s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.594216288 (211h39m11.553313519s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.596651299 (211h39m11.555748557s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.600004059 (211h39m11.559101306s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.619246354 (211h39m11.578343598s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.622590623 (211h39m11.581687861s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.633199644 (211h39m11.592296882s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.638567248 (211h39m11.597664486s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:05:18.402418747 (211h39m40.361515993s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: rpc: dial tcp 104.36.82.13:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402498626 (211h39m40.361595873s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402587513 (211h39m40.361684745s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402653738 (211h39m40.361750984s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402768644 (211h39m40.361865891s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402931362 (211h39m40.362028611s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402953112 (211h39m40.362050358s) [seg_pos={0 0}, upload=220354]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 13:05:18.518203938 (211h39m40.477301184s) [upload=220354]: Done uploading segments... Aug 17 13:05:18.518239058 (211h39m40.477336313s) [upload=220354]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.308167285s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...XXXX.....................................................] duration: 2.827281179s FAIL (2025-08-17 13:44:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.827341939s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [......X..........................................X.......X..] duration: 855.688996ms FAIL (2025-08-17 13:43:59 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:42:51.584264741 (212h17m13.543361997s) [upload=221021]: Starting upload Aug 17 13:42:51.584345151 (212h17m13.543442383s) [upload=221021]: Done starting upload Aug 17 13:42:51.584440425 (212h17m13.543537656s) [upload=221021]: Uploading segments... Aug 17 13:42:51.584508717 (212h17m13.543605973s) [upload=221021]: Got next segment. Inline: true Aug 17 13:42:51.584515874 (212h17m13.543613131s) [upload=221021]: Waiting for error group managing segments... Aug 17 13:42:51.584584133 (212h17m13.543681388s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:42:51.584590380 (212h17m13.543687637s) [upload=221021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:42:51.584594159 (212h17m13.543691419s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:42:51.784353099 (212h17m13.743450345s) [upload=221021]: Done uploading segments... Aug 17 13:42:51.784373595 (212h17m13.743470834s) [upload=221021]: Upload finished. err: <nil> Duration:* 855.740436ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [....XXXX....................................................] duration: 2.655340385s FAIL (2025-08-17 13:43:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.655370129s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....XXXX...................................................] duration: 2.510007906s FAIL (2025-08-17 13:43:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 13:04:48.671168571 (211h39m10.630265822s) [upload=220354]: Starting upload Aug 17 13:04:48.671289154 (211h39m10.630386405s) [upload=220354]: Done starting upload Aug 17 13:04:48.671375030 (211h39m10.630472267s) [upload=220354]: Uploading segments... Aug 17 13:04:48.671685291 (211h39m10.630782528s) [upload=220354]: Got next segment. Inline: false Aug 17 13:04:48.671859547 (211h39m10.630956792s) [upload=220354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:48.671868198 (211h39m10.630965451s) [upload=220354]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:48.784496066 (211h39m10.743593311s) [seg_pos={0 0}, upload=220354]: Begin upload segment... Aug 17 13:04:48.784546278 (211h39m10.743643523s) [seg_pos={0 0}, upload=220354]: Done begin upload segment. Aug 17 13:04:48.784576969 (211h39m10.743674214s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784591947 (211h39m10.743689244s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784926858 (211h39m10.744024104s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785024612 (211h39m10.744121858s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785177382 (211h39m10.744274627s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785386459 (211h39m10.744483708s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785426090 (211h39m10.744523335s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785675714 (211h39m10.744772959s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786030292 (211h39m10.745127541s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786370888 (211h39m10.745468139s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786497361 (211h39m10.745594613s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786709369 (211h39m10.745806618s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787016861 (211h39m10.746114097s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787272873 (211h39m10.746370129s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787671377 (211h39m10.746768629s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.788114039 (211h39m10.747211295s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:49.396438949 (211h39m11.355536206s) [upload=220354]: Next returned nil segment Aug 17 13:04:49.396459727 (211h39m11.355556984s) [upload=220354]: Waiting for error group managing segments... Aug 17 13:04:49.548390642 (211h39m11.507487888s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.555382402 (211h39m11.514479641s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.568461124 (211h39m11.527558355s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.594216288 (211h39m11.553313519s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.596651299 (211h39m11.555748557s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.600004059 (211h39m11.559101306s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.619246354 (211h39m11.578343598s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.622590623 (211h39m11.581687861s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.633199644 (211h39m11.592296882s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.638567248 (211h39m11.597664486s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:05:18.402418747 (211h39m40.361515993s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: rpc: dial tcp 104.36.82.13:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402498626 (211h39m40.361595873s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402587513 (211h39m40.361684745s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402653738 (211h39m40.361750984s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402768644 (211h39m40.361865891s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402931362 (211h39m40.362028611s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402953112 (211h39m40.362050358s) [seg_pos={0 0}, upload=220354]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 13:05:18.518203938 (211h39m40.477301184s) [upload=220354]: Done uploading segments... Aug 17 13:05:18.518239058 (211h39m40.477336313s) [upload=220354]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.510037897s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.......X..........................................X.......X.] duration: 860.321395ms FAIL (2025-08-17 13:43:24 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:42:51.584264741 (212h17m13.543361997s) [upload=221021]: Starting upload Aug 17 13:42:51.584345151 (212h17m13.543442383s) [upload=221021]: Done starting upload Aug 17 13:42:51.584440425 (212h17m13.543537656s) [upload=221021]: Uploading segments... Aug 17 13:42:51.584508717 (212h17m13.543605973s) [upload=221021]: Got next segment. Inline: true Aug 17 13:42:51.584515874 (212h17m13.543613131s) [upload=221021]: Waiting for error group managing segments... Aug 17 13:42:51.584584133 (212h17m13.543681388s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:42:51.584590380 (212h17m13.543687637s) [upload=221021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:42:51.584594159 (212h17m13.543691419s) [upload=221021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:42:51.784353099 (212h17m13.743450345s) [upload=221021]: Done uploading segments... Aug 17 13:42:51.784373595 (212h17m13.743470834s) [upload=221021]: Upload finished. err: <nil> Duration:* 860.354425ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [........XXX.X.........X.....XX..............................] duration: 2.673533852s FAIL (2025-08-17 13:43:20 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 2.673806443s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)