Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................XXXX...........................] duration: 2.809775215s FAIL (2025-08-17 13:26:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.809810747s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [.............................XXX.X.........X.....XX.........] duration: 2.718472323s FAIL (2025-08-17 13:26:44 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 2.718508604s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X..............................X...........................] duration: 1.033000235s FAIL (2025-08-17 13:26:31 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:05:53.292389310 (211h40m15.251486567s) [upload=220370]: Starting upload Aug 17 13:05:53.292477664 (211h40m15.251574912s) [upload=220370]: Done starting upload Aug 17 13:05:53.292569033 (211h40m15.251666269s) [upload=220370]: Uploading segments... Aug 17 13:05:53.292624407 (211h40m15.251721661s) [upload=220370]: Got next segment. Inline: true Aug 17 13:05:53.292630660 (211h40m15.251727906s) [upload=220370]: Waiting for error group managing segments... Aug 17 13:05:53.292698624 (211h40m15.25179587s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:53.292705778 (211h40m15.251803025s) [upload=220370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:05:53.292710869 (211h40m15.251808104s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:05:53.454358020 (211h40m15.413455265s) [upload=220370]: Done uploading segments... Aug 17 13:05:53.454376585 (211h40m15.413473816s) [upload=220370]: Upload finished. err: <nil> Duration:* 1.033028451s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................XXXX........................] duration: 2.359705041s FAIL (2025-08-17 13:25:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 13:04:48.671168571 (211h39m10.630265822s) [upload=220354]: Starting upload Aug 17 13:04:48.671289154 (211h39m10.630386405s) [upload=220354]: Done starting upload Aug 17 13:04:48.671375030 (211h39m10.630472267s) [upload=220354]: Uploading segments... Aug 17 13:04:48.671685291 (211h39m10.630782528s) [upload=220354]: Got next segment. Inline: false Aug 17 13:04:48.671859547 (211h39m10.630956792s) [upload=220354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:48.671868198 (211h39m10.630965451s) [upload=220354]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:48.784496066 (211h39m10.743593311s) [seg_pos={0 0}, upload=220354]: Begin upload segment... Aug 17 13:04:48.784546278 (211h39m10.743643523s) [seg_pos={0 0}, upload=220354]: Done begin upload segment. Aug 17 13:04:48.784576969 (211h39m10.743674214s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784591947 (211h39m10.743689244s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784926858 (211h39m10.744024104s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785024612 (211h39m10.744121858s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785177382 (211h39m10.744274627s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785386459 (211h39m10.744483708s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785426090 (211h39m10.744523335s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785675714 (211h39m10.744772959s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786030292 (211h39m10.745127541s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786370888 (211h39m10.745468139s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786497361 (211h39m10.745594613s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786709369 (211h39m10.745806618s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787016861 (211h39m10.746114097s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787272873 (211h39m10.746370129s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787671377 (211h39m10.746768629s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.788114039 (211h39m10.747211295s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:49.396438949 (211h39m11.355536206s) [upload=220354]: Next returned nil segment Aug 17 13:04:49.396459727 (211h39m11.355556984s) [upload=220354]: Waiting for error group managing segments... Aug 17 13:04:49.548390642 (211h39m11.507487888s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.555382402 (211h39m11.514479641s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.568461124 (211h39m11.527558355s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.594216288 (211h39m11.553313519s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.596651299 (211h39m11.555748557s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.600004059 (211h39m11.559101306s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.619246354 (211h39m11.578343598s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.622590623 (211h39m11.581687861s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.633199644 (211h39m11.592296882s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.638567248 (211h39m11.597664486s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:05:18.402418747 (211h39m40.361515993s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: rpc: dial tcp 104.36.82.13:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402498626 (211h39m40.361595873s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402587513 (211h39m40.361684745s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402653738 (211h39m40.361750984s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402768644 (211h39m40.361865891s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402931362 (211h39m40.362028611s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402953112 (211h39m40.362050358s) [seg_pos={0 0}, upload=220354]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 13:05:18.518203938 (211h39m40.477301184s) [upload=220354]: Done uploading segments... Aug 17 13:05:18.518239058 (211h39m40.477336313s) [upload=220354]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.359741285s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................XXXX..........................] duration: 2.711768474s FAIL (2025-08-17 13:25:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.711798401s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [..............................XXX.X.........X.....XX........] duration: 2.456461082s FAIL (2025-08-17 13:25:30 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 2.456507248s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..X..............................X..........................] duration: 1.009279477s FAIL (2025-08-17 13:25:19 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:05:53.292389310 (211h40m15.251486567s) [upload=220370]: Starting upload Aug 17 13:05:53.292477664 (211h40m15.251574912s) [upload=220370]: Done starting upload Aug 17 13:05:53.292569033 (211h40m15.251666269s) [upload=220370]: Uploading segments... Aug 17 13:05:53.292624407 (211h40m15.251721661s) [upload=220370]: Got next segment. Inline: true Aug 17 13:05:53.292630660 (211h40m15.251727906s) [upload=220370]: Waiting for error group managing segments... Aug 17 13:05:53.292698624 (211h40m15.25179587s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:53.292705778 (211h40m15.251803025s) [upload=220370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:05:53.292710869 (211h40m15.251808104s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:05:53.454358020 (211h40m15.413455265s) [upload=220370]: Done uploading segments... Aug 17 13:05:53.454376585 (211h40m15.413473816s) [upload=220370]: Upload finished. err: <nil> Duration:* 1.009338257s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................XXXX.......................] duration: 2.484876592s FAIL (2025-08-17 13:25:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 13:04:48.671168571 (211h39m10.630265822s) [upload=220354]: Starting upload Aug 17 13:04:48.671289154 (211h39m10.630386405s) [upload=220354]: Done starting upload Aug 17 13:04:48.671375030 (211h39m10.630472267s) [upload=220354]: Uploading segments... Aug 17 13:04:48.671685291 (211h39m10.630782528s) [upload=220354]: Got next segment. Inline: false Aug 17 13:04:48.671859547 (211h39m10.630956792s) [upload=220354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:48.671868198 (211h39m10.630965451s) [upload=220354]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:48.784496066 (211h39m10.743593311s) [seg_pos={0 0}, upload=220354]: Begin upload segment... Aug 17 13:04:48.784546278 (211h39m10.743643523s) [seg_pos={0 0}, upload=220354]: Done begin upload segment. Aug 17 13:04:48.784576969 (211h39m10.743674214s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784591947 (211h39m10.743689244s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784926858 (211h39m10.744024104s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785024612 (211h39m10.744121858s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785177382 (211h39m10.744274627s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785386459 (211h39m10.744483708s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785426090 (211h39m10.744523335s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785675714 (211h39m10.744772959s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786030292 (211h39m10.745127541s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786370888 (211h39m10.745468139s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786497361 (211h39m10.745594613s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786709369 (211h39m10.745806618s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787016861 (211h39m10.746114097s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787272873 (211h39m10.746370129s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787671377 (211h39m10.746768629s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.788114039 (211h39m10.747211295s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:49.396438949 (211h39m11.355536206s) [upload=220354]: Next returned nil segment Aug 17 13:04:49.396459727 (211h39m11.355556984s) [upload=220354]: Waiting for error group managing segments... Aug 17 13:04:49.548390642 (211h39m11.507487888s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.555382402 (211h39m11.514479641s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.568461124 (211h39m11.527558355s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.594216288 (211h39m11.553313519s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.596651299 (211h39m11.555748557s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.600004059 (211h39m11.559101306s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.619246354 (211h39m11.578343598s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.622590623 (211h39m11.581687861s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.633199644 (211h39m11.592296882s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.638567248 (211h39m11.597664486s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:05:18.402418747 (211h39m40.361515993s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: rpc: dial tcp 104.36.82.13:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402498626 (211h39m40.361595873s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402587513 (211h39m40.361684745s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402653738 (211h39m40.361750984s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402768644 (211h39m40.361865891s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402931362 (211h39m40.362028611s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402953112 (211h39m40.362050358s) [seg_pos={0 0}, upload=220354]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 13:05:18.518203938 (211h39m40.477301184s) [upload=220354]: Done uploading segments... Aug 17 13:05:18.518239058 (211h39m40.477336313s) [upload=220354]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.484916687s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................XXXX.........................] duration: 2.699627815s FAIL (2025-08-17 13:24:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.699714012s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [...............................XXX.X.........X.....XX.......] duration: 3.18064351s FAIL (2025-08-17 13:24:32 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 3.180673502s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................XXXX........................] duration: 2.471877943s FAIL (2025-08-17 13:24:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.471933933s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...X..............................X.........................] duration: 1.034347712s FAIL (2025-08-17 13:24:11 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:05:53.292389310 (211h40m15.251486567s) [upload=220370]: Starting upload Aug 17 13:05:53.292477664 (211h40m15.251574912s) [upload=220370]: Done starting upload Aug 17 13:05:53.292569033 (211h40m15.251666269s) [upload=220370]: Uploading segments... Aug 17 13:05:53.292624407 (211h40m15.251721661s) [upload=220370]: Got next segment. Inline: true Aug 17 13:05:53.292630660 (211h40m15.251727906s) [upload=220370]: Waiting for error group managing segments... Aug 17 13:05:53.292698624 (211h40m15.25179587s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:53.292705778 (211h40m15.251803025s) [upload=220370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:05:53.292710869 (211h40m15.251808104s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:05:53.454358020 (211h40m15.413455265s) [upload=220370]: Done uploading segments... Aug 17 13:05:53.454376585 (211h40m15.413473816s) [upload=220370]: Upload finished. err: <nil> Duration:* 1.034380583s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................XXXX......................] duration: 2.322255108s FAIL (2025-08-17 13:24:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 13:04:48.671168571 (211h39m10.630265822s) [upload=220354]: Starting upload Aug 17 13:04:48.671289154 (211h39m10.630386405s) [upload=220354]: Done starting upload Aug 17 13:04:48.671375030 (211h39m10.630472267s) [upload=220354]: Uploading segments... Aug 17 13:04:48.671685291 (211h39m10.630782528s) [upload=220354]: Got next segment. Inline: false Aug 17 13:04:48.671859547 (211h39m10.630956792s) [upload=220354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:48.671868198 (211h39m10.630965451s) [upload=220354]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:48.784496066 (211h39m10.743593311s) [seg_pos={0 0}, upload=220354]: Begin upload segment... Aug 17 13:04:48.784546278 (211h39m10.743643523s) [seg_pos={0 0}, upload=220354]: Done begin upload segment. Aug 17 13:04:48.784576969 (211h39m10.743674214s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784591947 (211h39m10.743689244s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784926858 (211h39m10.744024104s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785024612 (211h39m10.744121858s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785177382 (211h39m10.744274627s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785386459 (211h39m10.744483708s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785426090 (211h39m10.744523335s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785675714 (211h39m10.744772959s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786030292 (211h39m10.745127541s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786370888 (211h39m10.745468139s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786497361 (211h39m10.745594613s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786709369 (211h39m10.745806618s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787016861 (211h39m10.746114097s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787272873 (211h39m10.746370129s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787671377 (211h39m10.746768629s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.788114039 (211h39m10.747211295s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:49.396438949 (211h39m11.355536206s) [upload=220354]: Next returned nil segment Aug 17 13:04:49.396459727 (211h39m11.355556984s) [upload=220354]: Waiting for error group managing segments... Aug 17 13:04:49.548390642 (211h39m11.507487888s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.555382402 (211h39m11.514479641s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.568461124 (211h39m11.527558355s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.594216288 (211h39m11.553313519s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.596651299 (211h39m11.555748557s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.600004059 (211h39m11.559101306s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.619246354 (211h39m11.578343598s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.622590623 (211h39m11.581687861s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.633199644 (211h39m11.592296882s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.638567248 (211h39m11.597664486s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:05:18.402418747 (211h39m40.361515993s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: rpc: dial tcp 104.36.82.13:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402498626 (211h39m40.361595873s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402587513 (211h39m40.361684745s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402653738 (211h39m40.361750984s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402768644 (211h39m40.361865891s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402931362 (211h39m40.362028611s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402953112 (211h39m40.362050358s) [seg_pos={0 0}, upload=220354]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 13:05:18.518203938 (211h39m40.477301184s) [upload=220354]: Done uploading segments... Aug 17 13:05:18.518239058 (211h39m40.477336313s) [upload=220354]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.322290942s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................XXXX.......................] duration: 2.522226486s FAIL (2025-08-17 13:23:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.522326818s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................XXXX.....................] duration: 2.300088296s FAIL (2025-08-17 13:23:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 13:04:48.671168571 (211h39m10.630265822s) [upload=220354]: Starting upload Aug 17 13:04:48.671289154 (211h39m10.630386405s) [upload=220354]: Done starting upload Aug 17 13:04:48.671375030 (211h39m10.630472267s) [upload=220354]: Uploading segments... Aug 17 13:04:48.671685291 (211h39m10.630782528s) [upload=220354]: Got next segment. Inline: false Aug 17 13:04:48.671859547 (211h39m10.630956792s) [upload=220354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:48.671868198 (211h39m10.630965451s) [upload=220354]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:48.784496066 (211h39m10.743593311s) [seg_pos={0 0}, upload=220354]: Begin upload segment... Aug 17 13:04:48.784546278 (211h39m10.743643523s) [seg_pos={0 0}, upload=220354]: Done begin upload segment. Aug 17 13:04:48.784576969 (211h39m10.743674214s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784591947 (211h39m10.743689244s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784926858 (211h39m10.744024104s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785024612 (211h39m10.744121858s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785177382 (211h39m10.744274627s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785386459 (211h39m10.744483708s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785426090 (211h39m10.744523335s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785675714 (211h39m10.744772959s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786030292 (211h39m10.745127541s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786370888 (211h39m10.745468139s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786497361 (211h39m10.745594613s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786709369 (211h39m10.745806618s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787016861 (211h39m10.746114097s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787272873 (211h39m10.746370129s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787671377 (211h39m10.746768629s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.788114039 (211h39m10.747211295s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:49.396438949 (211h39m11.355536206s) [upload=220354]: Next returned nil segment Aug 17 13:04:49.396459727 (211h39m11.355556984s) [upload=220354]: Waiting for error group managing segments... Aug 17 13:04:49.548390642 (211h39m11.507487888s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.555382402 (211h39m11.514479641s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.568461124 (211h39m11.527558355s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.594216288 (211h39m11.553313519s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.596651299 (211h39m11.555748557s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.600004059 (211h39m11.559101306s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.619246354 (211h39m11.578343598s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.622590623 (211h39m11.581687861s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.633199644 (211h39m11.592296882s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.638567248 (211h39m11.597664486s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:05:18.402418747 (211h39m40.361515993s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: rpc: dial tcp 104.36.82.13:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402498626 (211h39m40.361595873s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402587513 (211h39m40.361684745s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402653738 (211h39m40.361750984s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402768644 (211h39m40.361865891s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402931362 (211h39m40.362028611s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402953112 (211h39m40.362050358s) [seg_pos={0 0}, upload=220354]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 13:05:18.518203938 (211h39m40.477301184s) [upload=220354]: Done uploading segments... Aug 17 13:05:18.518239058 (211h39m40.477336313s) [upload=220354]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.300157094s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....X..............................X........................] duration: 982.740479ms FAIL (2025-08-17 13:23:27 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:05:53.292389310 (211h40m15.251486567s) [upload=220370]: Starting upload Aug 17 13:05:53.292477664 (211h40m15.251574912s) [upload=220370]: Done starting upload Aug 17 13:05:53.292569033 (211h40m15.251666269s) [upload=220370]: Uploading segments... Aug 17 13:05:53.292624407 (211h40m15.251721661s) [upload=220370]: Got next segment. Inline: true Aug 17 13:05:53.292630660 (211h40m15.251727906s) [upload=220370]: Waiting for error group managing segments... Aug 17 13:05:53.292698624 (211h40m15.25179587s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:53.292705778 (211h40m15.251803025s) [upload=220370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:05:53.292710869 (211h40m15.251808104s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:05:53.454358020 (211h40m15.413455265s) [upload=220370]: Done uploading segments... Aug 17 13:05:53.454376585 (211h40m15.413473816s) [upload=220370]: Upload finished. err: <nil> Duration:* 982.769676ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [................................XXX.X.........X.....XX......] duration: 2.832314649s FAIL (2025-08-17 13:23:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 2.832374062s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....X..............................X.......................] duration: 901.384143ms FAIL (2025-08-17 13:22:50 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:05:53.292389310 (211h40m15.251486567s) [upload=220370]: Starting upload Aug 17 13:05:53.292477664 (211h40m15.251574912s) [upload=220370]: Done starting upload Aug 17 13:05:53.292569033 (211h40m15.251666269s) [upload=220370]: Uploading segments... Aug 17 13:05:53.292624407 (211h40m15.251721661s) [upload=220370]: Got next segment. Inline: true Aug 17 13:05:53.292630660 (211h40m15.251727906s) [upload=220370]: Waiting for error group managing segments... Aug 17 13:05:53.292698624 (211h40m15.25179587s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:53.292705778 (211h40m15.251803025s) [upload=220370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:05:53.292710869 (211h40m15.251808104s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:05:53.454358020 (211h40m15.413455265s) [upload=220370]: Done uploading segments... Aug 17 13:05:53.454376585 (211h40m15.413473816s) [upload=220370]: Upload finished. err: <nil> Duration:* 901.435479ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................XXXX......................] duration: 2.490453553s FAIL (2025-08-17 13:22:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.490485096s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................................XXXX....................] duration: 2.529970573s FAIL (2025-08-17 13:22:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 13:04:48.671168571 (211h39m10.630265822s) [upload=220354]: Starting upload Aug 17 13:04:48.671289154 (211h39m10.630386405s) [upload=220354]: Done starting upload Aug 17 13:04:48.671375030 (211h39m10.630472267s) [upload=220354]: Uploading segments... Aug 17 13:04:48.671685291 (211h39m10.630782528s) [upload=220354]: Got next segment. Inline: false Aug 17 13:04:48.671859547 (211h39m10.630956792s) [upload=220354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:48.671868198 (211h39m10.630965451s) [upload=220354]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:48.784496066 (211h39m10.743593311s) [seg_pos={0 0}, upload=220354]: Begin upload segment... Aug 17 13:04:48.784546278 (211h39m10.743643523s) [seg_pos={0 0}, upload=220354]: Done begin upload segment. Aug 17 13:04:48.784576969 (211h39m10.743674214s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784591947 (211h39m10.743689244s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784926858 (211h39m10.744024104s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785024612 (211h39m10.744121858s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785177382 (211h39m10.744274627s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785386459 (211h39m10.744483708s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785426090 (211h39m10.744523335s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785675714 (211h39m10.744772959s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786030292 (211h39m10.745127541s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786370888 (211h39m10.745468139s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786497361 (211h39m10.745594613s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786709369 (211h39m10.745806618s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787016861 (211h39m10.746114097s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787272873 (211h39m10.746370129s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787671377 (211h39m10.746768629s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.788114039 (211h39m10.747211295s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:49.396438949 (211h39m11.355536206s) [upload=220354]: Next returned nil segment Aug 17 13:04:49.396459727 (211h39m11.355556984s) [upload=220354]: Waiting for error group managing segments... Aug 17 13:04:49.548390642 (211h39m11.507487888s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.555382402 (211h39m11.514479641s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.568461124 (211h39m11.527558355s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.594216288 (211h39m11.553313519s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.596651299 (211h39m11.555748557s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.600004059 (211h39m11.559101306s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.619246354 (211h39m11.578343598s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.622590623 (211h39m11.581687861s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.633199644 (211h39m11.592296882s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.638567248 (211h39m11.597664486s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:05:18.402418747 (211h39m40.361515993s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: rpc: dial tcp 104.36.82.13:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402498626 (211h39m40.361595873s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402587513 (211h39m40.361684745s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402653738 (211h39m40.361750984s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402768644 (211h39m40.361865891s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402931362 (211h39m40.362028611s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402953112 (211h39m40.362050358s) [seg_pos={0 0}, upload=220354]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 13:05:18.518203938 (211h39m40.477301184s) [upload=220354]: Done uploading segments... Aug 17 13:05:18.518239058 (211h39m40.477336313s) [upload=220354]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.530001828s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................XXXX.....................] duration: 2.915266788s FAIL (2025-08-17 13:22:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.915319513s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [.................................XXX.X.........X.....XX.....] duration: 2.917919122s FAIL (2025-08-17 13:22:12 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 2.917970698s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......X..............................X......................] duration: 858.098791ms FAIL (2025-08-17 13:22:08 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:05:53.292389310 (211h40m15.251486567s) [upload=220370]: Starting upload Aug 17 13:05:53.292477664 (211h40m15.251574912s) [upload=220370]: Done starting upload Aug 17 13:05:53.292569033 (211h40m15.251666269s) [upload=220370]: Uploading segments... Aug 17 13:05:53.292624407 (211h40m15.251721661s) [upload=220370]: Got next segment. Inline: true Aug 17 13:05:53.292630660 (211h40m15.251727906s) [upload=220370]: Waiting for error group managing segments... Aug 17 13:05:53.292698624 (211h40m15.25179587s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:53.292705778 (211h40m15.251803025s) [upload=220370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:05:53.292710869 (211h40m15.251808104s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:05:53.454358020 (211h40m15.413455265s) [upload=220370]: Done uploading segments... Aug 17 13:05:53.454376585 (211h40m15.413473816s) [upload=220370]: Upload finished. err: <nil> Duration:* 858.138369ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................................XXXX....................] duration: 2.458088661s FAIL (2025-08-17 13:21:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.458121307s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......X..............................X.....................] duration: 965.462913ms FAIL (2025-08-17 13:21:40 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:05:53.292389310 (211h40m15.251486567s) [upload=220370]: Starting upload Aug 17 13:05:53.292477664 (211h40m15.251574912s) [upload=220370]: Done starting upload Aug 17 13:05:53.292569033 (211h40m15.251666269s) [upload=220370]: Uploading segments... Aug 17 13:05:53.292624407 (211h40m15.251721661s) [upload=220370]: Got next segment. Inline: true Aug 17 13:05:53.292630660 (211h40m15.251727906s) [upload=220370]: Waiting for error group managing segments... Aug 17 13:05:53.292698624 (211h40m15.25179587s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:53.292705778 (211h40m15.251803025s) [upload=220370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:05:53.292710869 (211h40m15.251808104s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:05:53.454358020 (211h40m15.413455265s) [upload=220370]: Done uploading segments... Aug 17 13:05:53.454376585 (211h40m15.413473816s) [upload=220370]: Upload finished. err: <nil> Duration:* 965.489352ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........X..............................X....................] duration: 1.000152801s FAIL (2025-08-17 13:21:20 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:05:53.292389310 (211h40m15.251486567s) [upload=220370]: Starting upload Aug 17 13:05:53.292477664 (211h40m15.251574912s) [upload=220370]: Done starting upload Aug 17 13:05:53.292569033 (211h40m15.251666269s) [upload=220370]: Uploading segments... Aug 17 13:05:53.292624407 (211h40m15.251721661s) [upload=220370]: Got next segment. Inline: true Aug 17 13:05:53.292630660 (211h40m15.251727906s) [upload=220370]: Waiting for error group managing segments... Aug 17 13:05:53.292698624 (211h40m15.25179587s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:53.292705778 (211h40m15.251803025s) [upload=220370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:05:53.292710869 (211h40m15.251808104s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:05:53.454358020 (211h40m15.413455265s) [upload=220370]: Done uploading segments... Aug 17 13:05:53.454376585 (211h40m15.413473816s) [upload=220370]: Upload finished. err: <nil> Duration:* 1.000187927s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................................XXXX...................] duration: 2.670221398s FAIL (2025-08-17 13:21:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.670254072s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................................XXXX...................] duration: 2.335009928s FAIL (2025-08-17 13:21:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 13:04:48.671168571 (211h39m10.630265822s) [upload=220354]: Starting upload Aug 17 13:04:48.671289154 (211h39m10.630386405s) [upload=220354]: Done starting upload Aug 17 13:04:48.671375030 (211h39m10.630472267s) [upload=220354]: Uploading segments... Aug 17 13:04:48.671685291 (211h39m10.630782528s) [upload=220354]: Got next segment. Inline: false Aug 17 13:04:48.671859547 (211h39m10.630956792s) [upload=220354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:48.671868198 (211h39m10.630965451s) [upload=220354]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:48.784496066 (211h39m10.743593311s) [seg_pos={0 0}, upload=220354]: Begin upload segment... Aug 17 13:04:48.784546278 (211h39m10.743643523s) [seg_pos={0 0}, upload=220354]: Done begin upload segment. Aug 17 13:04:48.784576969 (211h39m10.743674214s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784591947 (211h39m10.743689244s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784926858 (211h39m10.744024104s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785024612 (211h39m10.744121858s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785177382 (211h39m10.744274627s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785386459 (211h39m10.744483708s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785426090 (211h39m10.744523335s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785675714 (211h39m10.744772959s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786030292 (211h39m10.745127541s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786370888 (211h39m10.745468139s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786497361 (211h39m10.745594613s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786709369 (211h39m10.745806618s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787016861 (211h39m10.746114097s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787272873 (211h39m10.746370129s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787671377 (211h39m10.746768629s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.788114039 (211h39m10.747211295s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:49.396438949 (211h39m11.355536206s) [upload=220354]: Next returned nil segment Aug 17 13:04:49.396459727 (211h39m11.355556984s) [upload=220354]: Waiting for error group managing segments... Aug 17 13:04:49.548390642 (211h39m11.507487888s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.555382402 (211h39m11.514479641s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.568461124 (211h39m11.527558355s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.594216288 (211h39m11.553313519s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.596651299 (211h39m11.555748557s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.600004059 (211h39m11.559101306s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.619246354 (211h39m11.578343598s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.622590623 (211h39m11.581687861s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.633199644 (211h39m11.592296882s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.638567248 (211h39m11.597664486s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:05:18.402418747 (211h39m40.361515993s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: rpc: dial tcp 104.36.82.13:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402498626 (211h39m40.361595873s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402587513 (211h39m40.361684745s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402653738 (211h39m40.361750984s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402768644 (211h39m40.361865891s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402931362 (211h39m40.362028611s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402953112 (211h39m40.362050358s) [seg_pos={0 0}, upload=220354]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 13:05:18.518203938 (211h39m40.477301184s) [upload=220354]: Done uploading segments... Aug 17 13:05:18.518239058 (211h39m40.477336313s) [upload=220354]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.335045868s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [..................................XXX.X.........X.....XX....] duration: 2.469622358s FAIL (2025-08-17 13:21:05 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 2.469658153s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................................XXXX..................] duration: 2.664555286s FAIL (2025-08-17 13:20:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.664586647s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........X..............................X...................] duration: 825.61439ms FAIL (2025-08-17 13:20:47 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:05:53.292389310 (211h40m15.251486567s) [upload=220370]: Starting upload Aug 17 13:05:53.292477664 (211h40m15.251574912s) [upload=220370]: Done starting upload Aug 17 13:05:53.292569033 (211h40m15.251666269s) [upload=220370]: Uploading segments... Aug 17 13:05:53.292624407 (211h40m15.251721661s) [upload=220370]: Got next segment. Inline: true Aug 17 13:05:53.292630660 (211h40m15.251727906s) [upload=220370]: Waiting for error group managing segments... Aug 17 13:05:53.292698624 (211h40m15.25179587s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:53.292705778 (211h40m15.251803025s) [upload=220370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:05:53.292710869 (211h40m15.251808104s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:05:53.454358020 (211h40m15.413455265s) [upload=220370]: Done uploading segments... Aug 17 13:05:53.454376585 (211h40m15.413473816s) [upload=220370]: Upload finished. err: <nil> Duration:* 825.73739ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [...................................XXX.X.........X.....XX...] duration: 3.323622291s FAIL (2025-08-17 13:20:42 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 3.32365933s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........X..............................X..................] duration: 1.167527209s FAIL (2025-08-17 13:20:32 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:05:53.292389310 (211h40m15.251486567s) [upload=220370]: Starting upload Aug 17 13:05:53.292477664 (211h40m15.251574912s) [upload=220370]: Done starting upload Aug 17 13:05:53.292569033 (211h40m15.251666269s) [upload=220370]: Uploading segments... Aug 17 13:05:53.292624407 (211h40m15.251721661s) [upload=220370]: Got next segment. Inline: true Aug 17 13:05:53.292630660 (211h40m15.251727906s) [upload=220370]: Waiting for error group managing segments... Aug 17 13:05:53.292698624 (211h40m15.25179587s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:53.292705778 (211h40m15.251803025s) [upload=220370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:05:53.292710869 (211h40m15.251808104s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:05:53.454358020 (211h40m15.413455265s) [upload=220370]: Done uploading segments... Aug 17 13:05:53.454376585 (211h40m15.413473816s) [upload=220370]: Upload finished. err: <nil> Duration:* 1.167596308s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................XXXX.................] duration: 2.659482776s FAIL (2025-08-17 13:20:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.659522697s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................................XXXX..................] duration: 2.391060665s FAIL (2025-08-17 13:20:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 13:04:48.671168571 (211h39m10.630265822s) [upload=220354]: Starting upload Aug 17 13:04:48.671289154 (211h39m10.630386405s) [upload=220354]: Done starting upload Aug 17 13:04:48.671375030 (211h39m10.630472267s) [upload=220354]: Uploading segments... Aug 17 13:04:48.671685291 (211h39m10.630782528s) [upload=220354]: Got next segment. Inline: false Aug 17 13:04:48.671859547 (211h39m10.630956792s) [upload=220354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:48.671868198 (211h39m10.630965451s) [upload=220354]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:48.784496066 (211h39m10.743593311s) [seg_pos={0 0}, upload=220354]: Begin upload segment... Aug 17 13:04:48.784546278 (211h39m10.743643523s) [seg_pos={0 0}, upload=220354]: Done begin upload segment. Aug 17 13:04:48.784576969 (211h39m10.743674214s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784591947 (211h39m10.743689244s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784926858 (211h39m10.744024104s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785024612 (211h39m10.744121858s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785177382 (211h39m10.744274627s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785386459 (211h39m10.744483708s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785426090 (211h39m10.744523335s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785675714 (211h39m10.744772959s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786030292 (211h39m10.745127541s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786370888 (211h39m10.745468139s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786497361 (211h39m10.745594613s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786709369 (211h39m10.745806618s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787016861 (211h39m10.746114097s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787272873 (211h39m10.746370129s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787671377 (211h39m10.746768629s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.788114039 (211h39m10.747211295s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:49.396438949 (211h39m11.355536206s) [upload=220354]: Next returned nil segment Aug 17 13:04:49.396459727 (211h39m11.355556984s) [upload=220354]: Waiting for error group managing segments... Aug 17 13:04:49.548390642 (211h39m11.507487888s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.555382402 (211h39m11.514479641s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.568461124 (211h39m11.527558355s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.594216288 (211h39m11.553313519s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.596651299 (211h39m11.555748557s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.600004059 (211h39m11.559101306s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.619246354 (211h39m11.578343598s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.622590623 (211h39m11.581687861s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.633199644 (211h39m11.592296882s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.638567248 (211h39m11.597664486s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:05:18.402418747 (211h39m40.361515993s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: rpc: dial tcp 104.36.82.13:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402498626 (211h39m40.361595873s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402587513 (211h39m40.361684745s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402653738 (211h39m40.361750984s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402768644 (211h39m40.361865891s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402931362 (211h39m40.362028611s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402953112 (211h39m40.362050358s) [seg_pos={0 0}, upload=220354]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 13:05:18.518203938 (211h39m40.477301184s) [upload=220354]: Done uploading segments... Aug 17 13:05:18.518239058 (211h39m40.477336313s) [upload=220354]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.391099506s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [X..........X..............................X.................] duration: 1.067656076s FAIL (2025-08-17 13:20:06 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:05:53.292389310 (211h40m15.251486567s) [upload=220370]: Starting upload Aug 17 13:05:53.292477664 (211h40m15.251574912s) [upload=220370]: Done starting upload Aug 17 13:05:53.292569033 (211h40m15.251666269s) [upload=220370]: Uploading segments... Aug 17 13:05:53.292624407 (211h40m15.251721661s) [upload=220370]: Got next segment. Inline: true Aug 17 13:05:53.292630660 (211h40m15.251727906s) [upload=220370]: Waiting for error group managing segments... Aug 17 13:05:53.292698624 (211h40m15.25179587s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:53.292705778 (211h40m15.251803025s) [upload=220370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:05:53.292710869 (211h40m15.251808104s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:05:53.454358020 (211h40m15.413455265s) [upload=220370]: Done uploading segments... Aug 17 13:05:53.454376585 (211h40m15.413473816s) [upload=220370]: Upload finished. err: <nil> Duration:* 1.067685429s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [....................................XXX.X.........X.....XX..] duration: 2.518328195s FAIL (2025-08-17 13:19:39 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 2.518358209s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................XXXX................] duration: 2.923118093s FAIL (2025-08-17 13:19:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.923150404s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.X..........X..............................X................] duration: 1.048795566s FAIL (2025-08-17 13:19:20 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:05:53.292389310 (211h40m15.251486567s) [upload=220370]: Starting upload Aug 17 13:05:53.292477664 (211h40m15.251574912s) [upload=220370]: Done starting upload Aug 17 13:05:53.292569033 (211h40m15.251666269s) [upload=220370]: Uploading segments... Aug 17 13:05:53.292624407 (211h40m15.251721661s) [upload=220370]: Got next segment. Inline: true Aug 17 13:05:53.292630660 (211h40m15.251727906s) [upload=220370]: Waiting for error group managing segments... Aug 17 13:05:53.292698624 (211h40m15.25179587s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:53.292705778 (211h40m15.251803025s) [upload=220370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:05:53.292710869 (211h40m15.251808104s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:05:53.454358020 (211h40m15.413455265s) [upload=220370]: Done uploading segments... Aug 17 13:05:53.454376585 (211h40m15.413473816s) [upload=220370]: Upload finished. err: <nil> Duration:* 1.048849338s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....................................XXX.X.........X.....XX.] duration: 2.711826595s FAIL (2025-08-17 13:19:16 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 2.71185808s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................XXXX.................] duration: 2.268286754s FAIL (2025-08-17 13:18:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 13:04:48.671168571 (211h39m10.630265822s) [upload=220354]: Starting upload Aug 17 13:04:48.671289154 (211h39m10.630386405s) [upload=220354]: Done starting upload Aug 17 13:04:48.671375030 (211h39m10.630472267s) [upload=220354]: Uploading segments... Aug 17 13:04:48.671685291 (211h39m10.630782528s) [upload=220354]: Got next segment. Inline: false Aug 17 13:04:48.671859547 (211h39m10.630956792s) [upload=220354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:48.671868198 (211h39m10.630965451s) [upload=220354]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:48.784496066 (211h39m10.743593311s) [seg_pos={0 0}, upload=220354]: Begin upload segment... Aug 17 13:04:48.784546278 (211h39m10.743643523s) [seg_pos={0 0}, upload=220354]: Done begin upload segment. Aug 17 13:04:48.784576969 (211h39m10.743674214s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784591947 (211h39m10.743689244s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784926858 (211h39m10.744024104s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785024612 (211h39m10.744121858s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785177382 (211h39m10.744274627s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785386459 (211h39m10.744483708s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785426090 (211h39m10.744523335s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785675714 (211h39m10.744772959s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786030292 (211h39m10.745127541s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786370888 (211h39m10.745468139s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786497361 (211h39m10.745594613s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786709369 (211h39m10.745806618s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787016861 (211h39m10.746114097s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787272873 (211h39m10.746370129s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787671377 (211h39m10.746768629s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.788114039 (211h39m10.747211295s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:49.396438949 (211h39m11.355536206s) [upload=220354]: Next returned nil segment Aug 17 13:04:49.396459727 (211h39m11.355556984s) [upload=220354]: Waiting for error group managing segments... Aug 17 13:04:49.548390642 (211h39m11.507487888s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.555382402 (211h39m11.514479641s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.568461124 (211h39m11.527558355s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.594216288 (211h39m11.553313519s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.596651299 (211h39m11.555748557s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.600004059 (211h39m11.559101306s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.619246354 (211h39m11.578343598s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.622590623 (211h39m11.581687861s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.633199644 (211h39m11.592296882s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.638567248 (211h39m11.597664486s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:05:18.402418747 (211h39m40.361515993s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: rpc: dial tcp 104.36.82.13:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402498626 (211h39m40.361595873s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402587513 (211h39m40.361684745s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402653738 (211h39m40.361750984s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402768644 (211h39m40.361865891s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402931362 (211h39m40.362028611s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402953112 (211h39m40.362050358s) [seg_pos={0 0}, upload=220354]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 13:05:18.518203938 (211h39m40.477301184s) [upload=220354]: Done uploading segments... Aug 17 13:05:18.518239058 (211h39m40.477336313s) [upload=220354]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.268318036s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................................XXXX...............] duration: 2.488456852s FAIL (2025-08-17 13:18:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.488566846s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [......................................XXX.X.........X.....XX] duration: 1.813199051s FAIL (2025-08-17 13:18:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 13:18:35.864742071 (211h52m57.823839331s) [upload=220583]: Starting upload Aug 17 13:18:35.864845970 (211h52m57.823943223s) [upload=220583]: Done starting upload Aug 17 13:18:35.865003675 (211h52m57.824100913s) [upload=220583]: Uploading segments... Aug 17 13:18:35.865433012 (211h52m57.824530264s) [upload=220583]: Got next segment. Inline: false Aug 17 13:18:35.866067113 (211h52m57.825164365s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:18:35.866079521 (211h52m57.82517678s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:18:36.085890849 (211h52m58.044988105s) [seg_pos={0 0}, upload=220583]: Begin upload segment... Aug 17 13:18:36.086026111 (211h52m58.045123363s) [seg_pos={0 0}, upload=220583]: Done begin upload segment. Aug 17 13:18:36.086041115 (211h52m58.045138389s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086066225 (211h52m58.045163477s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086533162 (211h52m58.045630411s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.086684986 (211h52m58.045782239s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087238323 (211h52m58.046335574s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.087863059 (211h52m58.046960312s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090483300 (211h52m58.049580552s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.090839020 (211h52m58.049936275s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091248728 (211h52m58.050345977s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.091739067 (211h52m58.050836322s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092369228 (211h52m58.05146648s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.092830849 (211h52m58.051928095s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093170233 (211h52m58.052267481s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.093729080 (211h52m58.052826329s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.094159218 (211h52m58.05325647s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098255131 (211h52m58.057352384s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.098638202 (211h52m58.057735447s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.099035013 (211h52m58.058132269s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Uploading piece... Aug 17 13:18:36.656821064 (211h52m58.615918311s) [upload=220583]: Next returned nil segment Aug 17 13:18:36.656840063 (211h52m58.615937308s) [upload=220583]: Waiting for error group managing segments... Aug 17 13:18:36.800465071 (211h52m58.75956232s) [noise=true , address=77.90.31.11:4579 , piece_id=TMTRLPLED623JPIIL23FAD6Q3U6VFAU5P6MNMT4ECMZAARIJFICA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.800540759 (211h52m58.759637992s) [noise=true , address=104.36.82.11:4122 , piece_id=SSNKPJ3RGDWPJBX6PAZPCCZPQCF3JVR55SYJJJOYHP3BVKLTFQ6A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.808873581 (211h52m58.76797083s) [noise=true , address=104.36.82.58:4286 , piece_id=TXOQGN2XFEJHBNZNY56NH7MN3G6SIPUD2AJFRUDRWIM4FBBROP4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.826374819 (211h52m58.785472081s) [noise=true , address=79.127.219.48:4103 , piece_id=IMFBHWTOBL6PAVDFC6GQWVWODGPZTGVBDYSLT5BIWFG44AYUZLWQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.827144203 (211h52m58.786241434s) [noise=true , address=45.91.116.7:4038 , piece_id=U2FSB5GEPFTODBK6THDPTUXUX624CEI4EYHZERZIA2TRBVUEY4DA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834312143 (211h52m58.793409389s) [noise=true , address=104.36.82.44:4898 , piece_id=NWHARLK4X5FSU5KHYLYAMDVIQG2CDNZJ7N3MUXWJJUF6YY5GSXXA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.834696666 (211h52m58.793793917s) [noise=true , address=45.91.116.10:4092 , piece_id=AP6GEP7I4HNFZWDN6SBMSFPLMHZYVT2C6RICKJVTDTUL7JVA7Z7Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.843323208 (211h52m58.802420454s) [noise=true , address=104.36.82.60:4352 , piece_id=462HITT7NI26OF4DSH6RNN22DX3UG6IK4HCRBJMGAHBLUBD46G5A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860578644 (211h52m58.819675892s) [noise=true , address=104.36.82.59:4304 , piece_id=WRAJZAYRRXD7ZX6HGRE3CR7MY2MHNI2YRU7N2SLTYLF3YWB5ITTA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.860748922 (211h52m58.819846162s) [noise=true , address=79.127.219.38:4033 , piece_id=COJNC2PVUGF6VRRUAXWIT2SCV77KAJEAHWVSGJVWCIHCPJDZRKUQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925245200 (211h52m58.884342437s) [noise=true , address=77.90.31.14:4666 , piece_id=ULZMKJ3ZJNDNSM4PGFBFYL4RR3UUMVHPVEFQUXEAAC27OG3X4Q4A, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.925304416 (211h52m58.884401657s) [noise=true , address=104.36.82.17:4331 , piece_id=X24FA4DUNUVOABKFJK3HFMWGLM75RPKBMPRNRESDOP7SHFQ5YJCQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.934301283 (211h52m58.89339854s) [noise=true , address=77.90.31.10:4538 , piece_id=PZFHINPRLU7ZECCESODWFCJRMJBSWBNB4AP6HCWSYYDSPD65DAYQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977633822 (211h52m58.936731059s) [noise=true , address=104.36.82.23:4452 , piece_id=XJCVNLZIXRLZ3ZGYZF3XXWS2ET7HDZLSTFEIU6FLOJ5QXQUL4X3Q, seg_pos={0 0}, upload=220583]: Done uploading piece. err: <nil> Aug 17 13:18:36.977667742 (211h52m58.936764989s) [seg_pos={0 0}, upload=220583]: Segment reached optimal threshold of 14 pieces. Aug 17 13:18:36.977988257 (211h52m58.937085492s) [noise=true , address=66.23.193.175:4564 , piece_id=TTKAFI5I2IQVTPFXGZFBR4S6CXK3X2TH5FEQVBWAJ75SGP73WZIA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mHRT84eY5SYdo2c5LCZ4KsmXGsr6ErTqDCkehAYxqFJa58rfj): protocol: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50470->66.23.193.175:4564: use of closed network connection Aug 17 13:18:36.978040319 (211h52m58.937137573s) [noise=true , address=77.90.31.12:4623 , piece_id=52QO3OBAK5BVR5HVLO3CLCRO5ZEUOSOAB4BEJ6FFCYPTAREHYSHQ, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58092->77.90.31.12:4623: use of closed network connection Aug 17 13:18:36.978064968 (211h52m58.937162208s) [noise=true , address=104.36.82.5:4850 , piece_id=U6IHWVBYSHHIC4YC4TVOT2C2UQIFEXUFZIZENMNFYIWNUWKCLAQA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46920->104.36.82.5:4850: use of closed network connection Aug 17 13:18:36.978096020 (211h52m58.937193266s) [noise=true , address=77.90.31.21:4902 , piece_id=24MAKBL5XVKVVEE4SEPF4NMLE2CJX4OL5TOWZQCTBSJ4XOJQ3KGA, seg_pos={0 0}, upload=220583]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx): protocol: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52748->77.90.31.21:4902: use of closed network connection Aug 17 13:18:36.978114327 (211h52m58.937211566s) [seg_pos={0 0}, upload=220583]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 13:18:36.978191299 (211h52m58.937288544s) [upload=220583]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 13:18:36.978196381 (211h52m58.937293627s) [upload=220583]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:18:37.142431258 (211h52m59.101528494s) [upload=220583]: Done uploading segments... Aug 17 13:18:37.142453602 (211h52m59.101550837s) [upload=220583]: Upload finished. err: <nil> Duration: 1.813282625s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..X..........X..............................X...............] duration: 881.408359ms FAIL (2025-08-17 13:18:34 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:05:53.292389310 (211h40m15.251486567s) [upload=220370]: Starting upload Aug 17 13:05:53.292477664 (211h40m15.251574912s) [upload=220370]: Done starting upload Aug 17 13:05:53.292569033 (211h40m15.251666269s) [upload=220370]: Uploading segments... Aug 17 13:05:53.292624407 (211h40m15.251721661s) [upload=220370]: Got next segment. Inline: true Aug 17 13:05:53.292630660 (211h40m15.251727906s) [upload=220370]: Waiting for error group managing segments... Aug 17 13:05:53.292698624 (211h40m15.25179587s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:53.292705778 (211h40m15.251803025s) [upload=220370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:05:53.292710869 (211h40m15.251808104s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:05:53.454358020 (211h40m15.413455265s) [upload=220370]: Done uploading segments... Aug 17 13:05:53.454376585 (211h40m15.413473816s) [upload=220370]: Upload finished. err: <nil> Duration:* 881.450299ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................................XXXX..............] duration: 2.57474158s FAIL (2025-08-17 13:18:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.574799034s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................XXXX................] duration: 2.431833848s FAIL (2025-08-17 13:18:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 13:04:48.671168571 (211h39m10.630265822s) [upload=220354]: Starting upload Aug 17 13:04:48.671289154 (211h39m10.630386405s) [upload=220354]: Done starting upload Aug 17 13:04:48.671375030 (211h39m10.630472267s) [upload=220354]: Uploading segments... Aug 17 13:04:48.671685291 (211h39m10.630782528s) [upload=220354]: Got next segment. Inline: false Aug 17 13:04:48.671859547 (211h39m10.630956792s) [upload=220354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:48.671868198 (211h39m10.630965451s) [upload=220354]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:48.784496066 (211h39m10.743593311s) [seg_pos={0 0}, upload=220354]: Begin upload segment... Aug 17 13:04:48.784546278 (211h39m10.743643523s) [seg_pos={0 0}, upload=220354]: Done begin upload segment. Aug 17 13:04:48.784576969 (211h39m10.743674214s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784591947 (211h39m10.743689244s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784926858 (211h39m10.744024104s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785024612 (211h39m10.744121858s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785177382 (211h39m10.744274627s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785386459 (211h39m10.744483708s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785426090 (211h39m10.744523335s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785675714 (211h39m10.744772959s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786030292 (211h39m10.745127541s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786370888 (211h39m10.745468139s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786497361 (211h39m10.745594613s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786709369 (211h39m10.745806618s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787016861 (211h39m10.746114097s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787272873 (211h39m10.746370129s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787671377 (211h39m10.746768629s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.788114039 (211h39m10.747211295s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:49.396438949 (211h39m11.355536206s) [upload=220354]: Next returned nil segment Aug 17 13:04:49.396459727 (211h39m11.355556984s) [upload=220354]: Waiting for error group managing segments... Aug 17 13:04:49.548390642 (211h39m11.507487888s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.555382402 (211h39m11.514479641s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.568461124 (211h39m11.527558355s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.594216288 (211h39m11.553313519s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.596651299 (211h39m11.555748557s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.600004059 (211h39m11.559101306s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.619246354 (211h39m11.578343598s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.622590623 (211h39m11.581687861s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.633199644 (211h39m11.592296882s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.638567248 (211h39m11.597664486s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:05:18.402418747 (211h39m40.361515993s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: rpc: dial tcp 104.36.82.13:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402498626 (211h39m40.361595873s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402587513 (211h39m40.361684745s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402653738 (211h39m40.361750984s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402768644 (211h39m40.361865891s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402931362 (211h39m40.362028611s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402953112 (211h39m40.362050358s) [seg_pos={0 0}, upload=220354]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 13:05:18.518203938 (211h39m40.477301184s) [upload=220354]: Done uploading segments... Aug 17 13:05:18.518239058 (211h39m40.477336313s) [upload=220354]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.431947022s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.......................................XXX.X.........X.....X] duration: 359.50202ms FAIL (2025-08-17 13:17:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:17:53.851662115 (211h52m15.810759372s) [upload=220573]: Starting upload Aug 17 13:17:53.851736877 (211h52m15.810834131s) [upload=220573]: Done starting upload Aug 17 13:17:53.851905472 (211h52m15.811002734s) [upload=220573]: Uploading segments... Aug 17 13:17:53.852183876 (211h52m15.811281128s) [upload=220573]: Got next segment. Inline: false Aug 17 13:17:53.852239922 (211h52m15.811337172s) [upload=220573]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:17:53.852246971 (211h52m15.811344221s) [upload=220573]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:17:53.940398208 (211h52m15.899495462s) [upload=220573]: Next segment err: metaclient: Too Many Requests Aug 17 13:17:53.940434004 (211h52m15.899531251s) [upload=220573]: Done uploading segments... Aug 17 13:17:53.940454570 (211h52m15.899551817s) [upload=220573]: Upload finished. err: metaclient: Too Many Requests Duration: 359.525926ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...X..........X..............................X..............] duration: 821.947492ms FAIL (2025-08-17 13:17:53 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 13:05:53.292389310 (211h40m15.251486567s) [upload=220370]: Starting upload Aug 17 13:05:53.292477664 (211h40m15.251574912s) [upload=220370]: Done starting upload Aug 17 13:05:53.292569033 (211h40m15.251666269s) [upload=220370]: Uploading segments... Aug 17 13:05:53.292624407 (211h40m15.251721661s) [upload=220370]: Got next segment. Inline: true Aug 17 13:05:53.292630660 (211h40m15.251727906s) [upload=220370]: Waiting for error group managing segments... Aug 17 13:05:53.292698624 (211h40m15.25179587s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:53.292705778 (211h40m15.251803025s) [upload=220370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:05:53.292710869 (211h40m15.251808104s) [upload=220370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:05:53.454358020 (211h40m15.413455265s) [upload=220370]: Done uploading segments... Aug 17 13:05:53.454376585 (211h40m15.413473816s) [upload=220370]: Upload finished. err: <nil> Duration:* 821.981905ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................................XXXX...............] duration: 2.332494264s FAIL (2025-08-17 13:17:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 13:04:48.671168571 (211h39m10.630265822s) [upload=220354]: Starting upload Aug 17 13:04:48.671289154 (211h39m10.630386405s) [upload=220354]: Done starting upload Aug 17 13:04:48.671375030 (211h39m10.630472267s) [upload=220354]: Uploading segments... Aug 17 13:04:48.671685291 (211h39m10.630782528s) [upload=220354]: Got next segment. Inline: false Aug 17 13:04:48.671859547 (211h39m10.630956792s) [upload=220354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:48.671868198 (211h39m10.630965451s) [upload=220354]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:48.784496066 (211h39m10.743593311s) [seg_pos={0 0}, upload=220354]: Begin upload segment... Aug 17 13:04:48.784546278 (211h39m10.743643523s) [seg_pos={0 0}, upload=220354]: Done begin upload segment. Aug 17 13:04:48.784576969 (211h39m10.743674214s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784591947 (211h39m10.743689244s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.784926858 (211h39m10.744024104s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785024612 (211h39m10.744121858s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785177382 (211h39m10.744274627s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785386459 (211h39m10.744483708s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785426090 (211h39m10.744523335s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.785675714 (211h39m10.744772959s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786030292 (211h39m10.745127541s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786370888 (211h39m10.745468139s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786497361 (211h39m10.745594613s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.786709369 (211h39m10.745806618s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787016861 (211h39m10.746114097s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787272873 (211h39m10.746370129s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.787671377 (211h39m10.746768629s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:48.788114039 (211h39m10.747211295s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Uploading piece... Aug 17 13:04:49.396438949 (211h39m11.355536206s) [upload=220354]: Next returned nil segment Aug 17 13:04:49.396459727 (211h39m11.355556984s) [upload=220354]: Waiting for error group managing segments... Aug 17 13:04:49.548390642 (211h39m11.507487888s) [noise=true , address=77.90.31.11:4553 , piece_id=AG7MIKIR6GZIAHY7MHORQRCNQCYR7IFHMFPTXAU54UJME5JPAXZA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.555382402 (211h39m11.514479641s) [noise=true , address=45.91.116.11:4118 , piece_id=FMR757UBDMQR6KY5NXBJDCIEJWVMVDXZVKKQ6WJSCB6QJPXSMFMQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.568461124 (211h39m11.527558355s) [noise=true , address=45.91.116.8:4047 , piece_id=N2MAQURNFGRJODRX7NNIWR7FBWMBY6OEIMS67YDRU37NJBIGQEFQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.594216288 (211h39m11.553313519s) [noise=true , address=45.91.116.4:4940 , piece_id=2WVMQJUCVS4RISINDJVSNLUPEQTM5A5UUVEKH4LBPUNV37OZ4GHQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.596651299 (211h39m11.555748557s) [noise=true , address=45.91.116.14:4189 , piece_id=PZOLHTZEUD5ZBOPZZCMYSPBHBF3HYIQOJIQQDWKW3BXQBH35KOPQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.600004059 (211h39m11.559101306s) [noise=true , address=45.91.116.5:4297 , piece_id=KGKAAJRVI4UUO7GL36NKH55IPLZM7CYBNW2D632OTIW2XRCXFL4A, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.619246354 (211h39m11.578343598s) [noise=true , address=45.91.116.6:4974 , piece_id=D2UBLOQZFV5Y65PPNOYZANFQYJA3VW5MYHROACSEDU3N4KBM2ECQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.622590623 (211h39m11.581687861s) [noise=true , address=45.91.116.5:4299 , piece_id=C7VTE7JWLJJNPIE7FFFUXR46VFSIQKI7OJJWWAIXBDHCKPTFMGOQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.633199644 (211h39m11.592296882s) [noise=true , address=77.90.31.13:4628 , piece_id=VUCK3GUU7RKFAYILIYPNGHCZAXYK743ZBN35LVBZMGZGLEM73X5Q, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:04:49.638567248 (211h39m11.597664486s) [noise=true , address=77.90.31.21:4879 , piece_id=NIQXMIEXQFZZ42M5MZACGECP5ZBR32MSZGHNFHJ6QENFN2DWEZSQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: <nil> Aug 17 13:05:18.402418747 (211h39m40.361515993s) [noise=true , address=104.36.82.13:4186 , piece_id=RPF3NP62OFMDLTDNL5QIBV2IB2SM5AGXSG32QWDGMFGRSNLFACCA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: rpc: dial tcp 104.36.82.13:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402498626 (211h39m40.361595873s) [noise=true , address=104.36.82.4:4801 , piece_id=U4NVLZC6LLZFIZE3GNN72R5E7JVPPDR66FEJZ5QCRBIICLPBKNHA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:18.402587513 (211h39m40.361684745s) [noise=true , address=104.36.82.8:4969 , piece_id=QBKZULXXBNFTBZE3GRFX6WKLFS4B3UEZDN6D4O3T74AO5R5JR5EA, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402653738 (211h39m40.361750984s) [noise=true , address=104.36.82.7:4909 , piece_id=MZKLM66QYEALDLPVAECEMRFNTCLV3SXOSNHMJB5CKMPMBLHE4AEQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402768644 (211h39m40.361865891s) [noise=true , address=104.36.82.11:4114 , piece_id=72DKJ6BLHEEOEMMUI3NLYYYRUUABEXDOJK2I3Y3M3UGEWMGDR3SQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402931362 (211h39m40.362028611s) [noise=true , address=104.36.82.2:4730 , piece_id=6IMVJQPWAS7URN7QYWPYCD2RZK2C5GZCQGUSJ6RUOL3XJO3O7HZQ, seg_pos={0 0}, upload=220354]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:05:18.402953112 (211h39m40.362050358s) [seg_pos={0 0}, upload=220354]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 13:05:18.518203938 (211h39m40.477301184s) [upload=220354]: Done uploading segments... Aug 17 13:05:18.518239058 (211h39m40.477336313s) [upload=220354]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.332575459s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................................XXXX.............] duration: 2.503043385s FAIL (2025-08-17 13:17:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== s={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860424250 (211h39m24.819521505s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860443782 (211h39m24.819541042s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860834658 (211h39m24.819932015s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.860891189 (211h39m24.819988472s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861295596 (211h39m24.820392854s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861308085 (211h39m24.820405343s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861721034 (211h39m24.820818312s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.861738713 (211h39m24.82083598s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862205183 (211h39m24.821302442s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862325407 (211h39m24.821422696s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862620160 (211h39m24.82171753s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:02.862835186 (211h39m24.821932443s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Uploading piece... Aug 17 13:05:03.436772519 (211h39m25.395869766s) [upload=220356]: Next returned nil segment Aug 17 13:05:03.436799550 (211h39m25.395896811s) [upload=220356]: Waiting for error group managing segments... Aug 17 13:05:03.580591291 (211h39m25.539688524s) [noise=true , address=45.91.116.12:4228 , piece_id=MTH3FCTHOBUONI3ATWGLX7T422COVZAR44ES2SASV6IPS4TT32NA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.649197127 (211h39m25.608294368s) [noise=true , address=77.90.31.15:4703 , piece_id=P7LMDBYDZ72IDHBPVYMAUKQAF3PEHYZ7TWL72VGCNLBDF56GAU4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.678085375 (211h39m25.637182615s) [noise=true , address=77.90.31.14:4676 , piece_id=XTWEFWPNQHGH3HBL42U2JVOYHI34SZQ3BAJBUW5OHAYQG73J7N6Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.690242999 (211h39m25.649340239s) [noise=true , address=77.90.31.11:4563 , piece_id=SMDGS5CXFE6LF7CAEENVWLSWTSXLBOBX7QKPZVXFWRHD4I6VC7GA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:03.948496632 (211h39m25.907593866s) [noise=true , address=79.127.201.215:4354 , piece_id=RHT2PSXNWPW6DX5FKBGITLOFMJ36KMCFRHV6AP5MHIGQCYIXR6NQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:05:04.108115883 (211h39m26.067213141s) [noise=true , address=66.23.193.175:4555 , piece_id=6EKCGUSQSLFHFXNUL6BDMMG7P2W4M3ETJ4D5XNJC5V4QHYLS37UQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: <nil> Aug 17 13:06:02.389131075 (211h40m24.348228332s) [noise=true , address=104.36.82.4:4821 , piece_id=SZQ4ODJ6IBPXZPPLMZZA7533MZBN64C5NQ5S2NHNYPFQVDNAJ6LQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:104.36.82.4:4821): protocol: rpc: dial tcp 104.36.82.4:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389262413 (211h40m24.348359673s) [noise=true , address=104.36.82.39:4756 , piece_id=XCWABRWTJ4MGGZKVMEYEFX5TEQELEPN6YBA33HVVU75VF2Z6ZZPQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:104.36.82.39:4756): protocol: rpc: dial tcp 104.36.82.39:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389582130 (211h40m24.348679388s) [noise=true , address=104.36.82.51:4079 , piece_id=4JGPKABKRRQU4DAIMRLQZAFRHA7OS45MFYQBTA6TDQTSXKRSKFPA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: write tcp 139.59.144.22:46906->104.36.82.51:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389643456 (211h40m24.348740693s) [noise=true , address=104.36.82.21:4506 , piece_id=IO7ZU2CMWODCGXUKHA2NORHMNAZCD24RWEZEKYEIZIN24CK6XLRQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389692881 (211h40m24.348790138s) [noise=true , address=104.36.82.23:4462 , piece_id=3ML37MP2TQYQDRUDACGK3LVMZPY6CQ6CDETI3UCHRUQ2EOOPHXJA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389734487 (211h40m24.348831728s) [noise=true , address=104.36.82.2:4762 , piece_id=BTP6XPRTEVYKQEXKZYROAV7WF3GPMGPDWCJNCYBR4SBTINE3JYXA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: write tcp 139.59.144.22:36810->104.36.82.2:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389842740 (211h40m24.348939975s) [noise=true , address=104.36.82.60:4352 , piece_id=LYTVTLNMO7DQX6F3ZSEPWROQTJPQRST4IAXSNH24SCHU4QNX4WCA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi, address:104.36.82.60:4352): protocol: write tcp 139.59.144.22:42132->104.36.82.60:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389939930 (211h40m24.349037163s) [noise=true , address=104.36.82.40:4949 , piece_id=JO4YEFXT72PQESTS5XAJRB5744GKOQIUB2L2QVA7HM6KML3MFN7A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.389986244 (211h40m24.349083492s) [noise=true , address=104.36.82.58:4275 , piece_id=FONUFYYHTF5A5OAMTO2JH4XLWWKTXR4MFN6MYKFCIJ4EPFII3P4Q, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12Mb3s64JoVf1TbMuMRpT67xzAN4YSE9txxRnjomuvZUpSj9tt3, address:104.36.82.58:4275): protocol: write tcp 139.59.144.22:34692->104.36.82.58:4275: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390006984 (211h40m24.349104216s) [noise=true , address=104.36.82.36:4626 , piece_id=72QP3R24SKEOGZAY2FJPMVYNQVMIKAEFNZ7TZRD2X4RAR5WHJE5A, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390056341 (211h40m24.34915358s) [noise=true , address=104.36.82.5:4844 , piece_id=S6FQV3NHEMEUQFH7ARLE76XJJBQ4WE22ZGI3CPLWGFE3QBYCNOTA, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: write tcp 139.59.144.22:34632->104.36.82.5:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390105755 (211h40m24.349203011s) [noise=true , address=104.36.82.9:4025 , piece_id=I6MSLKGJY5N2CKRVKAKXVG6UAZN65CBQHYV6DUU5IGY5SVZZVNDQ, seg_pos={0 0}, upload=220356]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 139.59.144.22:37148->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:02.390162167 (211h40m24.349259427s) [seg_pos={0 0}, upload=220356]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 17 13:06:02.490382093 (211h40m24.449479331s) [upload=220356]: Done uploading segments... Aug 17 13:06:02.490422033 (211h40m24.449519267s) [upload=220356]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 2.503076718s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)