Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.271994598s FAIL (2025-08-09 12:27:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 2.272078827s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.370243805s FAIL (2025-08-09 12:27:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 2.370307233s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 1.627048664s FAIL (2025-08-09 12:26:50 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 9 12:22:07.435852398 (18h56m29.394949665s) [upload=19731]: Starting upload Aug 9 12:22:07.435936038 (18h56m29.395033284s) [upload=19731]: Done starting upload Aug 9 12:22:07.436079460 (18h56m29.395176706s) [upload=19731]: Uploading segments... Aug 9 12:22:07.436156695 (18h56m29.395253951s) [upload=19731]: Got next segment. Inline: true Aug 9 12:22:07.436164855 (18h56m29.395262102s) [upload=19731]: Waiting for error group managing segments... Aug 9 12:22:07.436239336 (18h56m29.395336567s) [upload=19731]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:22:07.436245387 (18h56m29.395342637s) [upload=19731]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:22:07.436249733 (18h56m29.395346981s) [upload=19731]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:22:07.588907013 (18h56m29.548004279s) [upload=19731]: Done uploading segments... Aug 9 12:22:07.588928061 (18h56m29.5480253s) [upload=19731]: Upload finished. err: <nil> Duration:* 1.62709072s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.368793604s FAIL (2025-08-09 12:26:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 2.368833705s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 893.702659ms FAIL (2025-08-09 12:26:11 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 9 12:22:07.435852398 (18h56m29.394949665s) [upload=19731]: Starting upload Aug 9 12:22:07.435936038 (18h56m29.395033284s) [upload=19731]: Done starting upload Aug 9 12:22:07.436079460 (18h56m29.395176706s) [upload=19731]: Uploading segments... Aug 9 12:22:07.436156695 (18h56m29.395253951s) [upload=19731]: Got next segment. Inline: true Aug 9 12:22:07.436164855 (18h56m29.395262102s) [upload=19731]: Waiting for error group managing segments... Aug 9 12:22:07.436239336 (18h56m29.395336567s) [upload=19731]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:22:07.436245387 (18h56m29.395342637s) [upload=19731]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:22:07.436249733 (18h56m29.395346981s) [upload=19731]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:22:07.588907013 (18h56m29.548004279s) [upload=19731]: Done uploading segments... Aug 9 12:22:07.588928061 (18h56m29.5480253s) [upload=19731]: Upload finished. err: <nil> Duration:* 893.738924ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.249133473s FAIL (2025-08-09 12:25:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 2.249166167s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 956.163891ms FAIL (2025-08-09 12:25:04 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 9 12:22:07.435852398 (18h56m29.394949665s) [upload=19731]: Starting upload Aug 9 12:22:07.435936038 (18h56m29.395033284s) [upload=19731]: Done starting upload Aug 9 12:22:07.436079460 (18h56m29.395176706s) [upload=19731]: Uploading segments... Aug 9 12:22:07.436156695 (18h56m29.395253951s) [upload=19731]: Got next segment. Inline: true Aug 9 12:22:07.436164855 (18h56m29.395262102s) [upload=19731]: Waiting for error group managing segments... Aug 9 12:22:07.436239336 (18h56m29.395336567s) [upload=19731]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:22:07.436245387 (18h56m29.395342637s) [upload=19731]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:22:07.436249733 (18h56m29.395346981s) [upload=19731]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:22:07.588907013 (18h56m29.548004279s) [upload=19731]: Done uploading segments... Aug 9 12:22:07.588928061 (18h56m29.5480253s) [upload=19731]: Upload finished. err: <nil> Duration:* 956.190233ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 940.428714ms FAIL (2025-08-09 12:24:48 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 9 12:22:07.435852398 (18h56m29.394949665s) [upload=19731]: Starting upload Aug 9 12:22:07.435936038 (18h56m29.395033284s) [upload=19731]: Done starting upload Aug 9 12:22:07.436079460 (18h56m29.395176706s) [upload=19731]: Uploading segments... Aug 9 12:22:07.436156695 (18h56m29.395253951s) [upload=19731]: Got next segment. Inline: true Aug 9 12:22:07.436164855 (18h56m29.395262102s) [upload=19731]: Waiting for error group managing segments... Aug 9 12:22:07.436239336 (18h56m29.395336567s) [upload=19731]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:22:07.436245387 (18h56m29.395342637s) [upload=19731]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:22:07.436249733 (18h56m29.395346981s) [upload=19731]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:22:07.588907013 (18h56m29.548004279s) [upload=19731]: Done uploading segments... Aug 9 12:22:07.588928061 (18h56m29.5480253s) [upload=19731]: Upload finished. err: <nil> Duration:* 940.472886ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.412284324s FAIL (2025-08-09 12:24:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 2.412331413s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.381053562s FAIL (2025-08-09 12:24:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 2.381088183s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 836.824567ms FAIL (2025-08-09 12:23:58 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 9 12:22:07.435852398 (18h56m29.394949665s) [upload=19731]: Starting upload Aug 9 12:22:07.435936038 (18h56m29.395033284s) [upload=19731]: Done starting upload Aug 9 12:22:07.436079460 (18h56m29.395176706s) [upload=19731]: Uploading segments... Aug 9 12:22:07.436156695 (18h56m29.395253951s) [upload=19731]: Got next segment. Inline: true Aug 9 12:22:07.436164855 (18h56m29.395262102s) [upload=19731]: Waiting for error group managing segments... Aug 9 12:22:07.436239336 (18h56m29.395336567s) [upload=19731]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:22:07.436245387 (18h56m29.395342637s) [upload=19731]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:22:07.436249733 (18h56m29.395346981s) [upload=19731]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:22:07.588907013 (18h56m29.548004279s) [upload=19731]: Done uploading segments... Aug 9 12:22:07.588928061 (18h56m29.5480253s) [upload=19731]: Upload finished. err: <nil> Duration:* 836.929301ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 895.954353ms FAIL (2025-08-09 12:23:39 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 9 12:22:07.435852398 (18h56m29.394949665s) [upload=19731]: Starting upload Aug 9 12:22:07.435936038 (18h56m29.395033284s) [upload=19731]: Done starting upload Aug 9 12:22:07.436079460 (18h56m29.395176706s) [upload=19731]: Uploading segments... Aug 9 12:22:07.436156695 (18h56m29.395253951s) [upload=19731]: Got next segment. Inline: true Aug 9 12:22:07.436164855 (18h56m29.395262102s) [upload=19731]: Waiting for error group managing segments... Aug 9 12:22:07.436239336 (18h56m29.395336567s) [upload=19731]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:22:07.436245387 (18h56m29.395342637s) [upload=19731]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:22:07.436249733 (18h56m29.395346981s) [upload=19731]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:22:07.588907013 (18h56m29.548004279s) [upload=19731]: Done uploading segments... Aug 9 12:22:07.588928061 (18h56m29.5480253s) [upload=19731]: Upload finished. err: <nil> Duration:* 896.001751ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.296932344s FAIL (2025-08-09 12:23:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 2.296972122s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.326984781s FAIL (2025-08-09 12:23:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 2.327013569s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 921.452796ms FAIL (2025-08-09 12:23:00 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 9 12:22:07.435852398 (18h56m29.394949665s) [upload=19731]: Starting upload Aug 9 12:22:07.435936038 (18h56m29.395033284s) [upload=19731]: Done starting upload Aug 9 12:22:07.436079460 (18h56m29.395176706s) [upload=19731]: Uploading segments... Aug 9 12:22:07.436156695 (18h56m29.395253951s) [upload=19731]: Got next segment. Inline: true Aug 9 12:22:07.436164855 (18h56m29.395262102s) [upload=19731]: Waiting for error group managing segments... Aug 9 12:22:07.436239336 (18h56m29.395336567s) [upload=19731]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:22:07.436245387 (18h56m29.395342637s) [upload=19731]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:22:07.436249733 (18h56m29.395346981s) [upload=19731]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:22:07.588907013 (18h56m29.548004279s) [upload=19731]: Done uploading segments... Aug 9 12:22:07.588928061 (18h56m29.5480253s) [upload=19731]: Upload finished. err: <nil> Duration:* 921.548946ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X..............................................X............] duration: 2.459024725s FAIL (2025-08-09 12:22:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 2.459066235s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 833.723701ms FAIL (2025-08-09 12:22:31 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 9 12:22:07.435852398 (18h56m29.394949665s) [upload=19731]: Starting upload Aug 9 12:22:07.435936038 (18h56m29.395033284s) [upload=19731]: Done starting upload Aug 9 12:22:07.436079460 (18h56m29.395176706s) [upload=19731]: Uploading segments... Aug 9 12:22:07.436156695 (18h56m29.395253951s) [upload=19731]: Got next segment. Inline: true Aug 9 12:22:07.436164855 (18h56m29.395262102s) [upload=19731]: Waiting for error group managing segments... Aug 9 12:22:07.436239336 (18h56m29.395336567s) [upload=19731]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:22:07.436245387 (18h56m29.395342637s) [upload=19731]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:22:07.436249733 (18h56m29.395346981s) [upload=19731]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:22:07.588907013 (18h56m29.548004279s) [upload=19731]: Done uploading segments... Aug 9 12:22:07.588928061 (18h56m29.5480253s) [upload=19731]: Upload finished. err: <nil> Duration:* 833.765451ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 722.825506ms FAIL (2025-08-09 12:22:07 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 9 12:22:07.435852398 (18h56m29.394949665s) [upload=19731]: Starting upload Aug 9 12:22:07.435936038 (18h56m29.395033284s) [upload=19731]: Done starting upload Aug 9 12:22:07.436079460 (18h56m29.395176706s) [upload=19731]: Uploading segments... Aug 9 12:22:07.436156695 (18h56m29.395253951s) [upload=19731]: Got next segment. Inline: true Aug 9 12:22:07.436164855 (18h56m29.395262102s) [upload=19731]: Waiting for error group managing segments... Aug 9 12:22:07.436239336 (18h56m29.395336567s) [upload=19731]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:22:07.436245387 (18h56m29.395342637s) [upload=19731]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:22:07.436249733 (18h56m29.395346981s) [upload=19731]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:22:07.588907013 (18h56m29.548004279s) [upload=19731]: Done uploading segments... Aug 9 12:22:07.588928061 (18h56m29.5480253s) [upload=19731]: Upload finished. err: <nil> Duration:* 722.879881ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X..............................................X...........] duration: 2.472279709s FAIL (2025-08-09 12:22:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 2.472327517s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X..............................................X..........] duration: 2.257483134s FAIL (2025-08-09 12:21:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 2.257627896s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X..............................................X.........] duration: 2.440610711s FAIL (2025-08-09 12:20:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 2.440642533s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X..............................................X........] duration: 2.383131666s FAIL (2025-08-09 12:19:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 2.383164568s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 945.109579ms FAIL (2025-08-09 12:19:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 945.136237ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 826.731568ms FAIL (2025-08-09 12:18:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 826.766278ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X..............................................X.......] duration: 2.357711523s FAIL (2025-08-09 12:18:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 2.357775199s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 918.334542ms FAIL (2025-08-09 12:18:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 918.365981ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X..............................................X......] duration: 2.232068739s FAIL (2025-08-09 12:17:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 2.232098329s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X..............................................X.....] duration: 2.440560447s FAIL (2025-08-09 12:17:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 2.440593192s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 874.049474ms FAIL (2025-08-09 12:16:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 874.078102ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 1.151941034s FAIL (2025-08-09 12:15:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 1.151971097s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X..............................................X....] duration: 2.281767079s FAIL (2025-08-09 12:15:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 2.281801947s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 1.01715687s FAIL (2025-08-09 12:15:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 1.01719933s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X..............................................X...] duration: 2.318499173s FAIL (2025-08-09 12:14:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 2.318593181s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 1.056165741s FAIL (2025-08-09 12:14:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 1.0561927s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.722324204s FAIL (2025-08-09 12:14:06 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 2.722356912s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X..............................................X..] duration: 2.400662295s FAIL (2025-08-09 12:13:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 2.400700722s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.469251922s FAIL (2025-08-09 12:13:22 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 2.469282729s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 960.38648ms FAIL (2025-08-09 12:13:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 960.410595ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 960.521465ms FAIL (2025-08-09 12:13:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 960.5996ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X..............................................X.] duration: 2.30140229s FAIL (2025-08-09 12:12:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 2.301441254s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.833360748s FAIL (2025-08-09 12:12:38 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 2.833394033s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 1.032423558s FAIL (2025-08-09 12:12:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 1.032461551s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............X..............................................X] duration: 30.10739907s FAIL (2025-08-09 12:12:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== GOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.909653192 (18h46m9.868750457s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910032272 (18h46m9.869129528s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910049455 (18h46m9.869146687s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910346934 (18h46m9.86944419s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:47.910697198 (18h46m9.869794451s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Uploading piece... Aug 9 12:11:48.566764084 (18h46m10.525861316s) [upload=19553]: Next returned nil segment Aug 9 12:11:48.567019735 (18h46m10.526116983s) [upload=19553]: Waiting for error group managing segments... Aug 9 12:11:48.710140699 (18h46m10.669237938s) [noise=true , address=104.36.82.10:4078 , piece_id=LC6KTKPFFAZVQLJ3JNJ6VTCFZPOFOZQZWFU35SREBYMQVOABK4CQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:15.936319677 (18h46m37.895417003s) [noise=true , address=77.90.31.18:4806 , piece_id=LBXUNKIXCVZOPOWRVGZ6LHZD5ZVB6JJGCNDH7C56DLEHEVIEXIAA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: <nil> Aug 9 12:12:17.518388239 (18h46m39.477485484s) [noise=true , address=77.90.31.11:4553 , piece_id=PMVGCBUFZDVYK5QGLEFJWZZJTA5T35MTGGTXUHNI523UOTWJGQKA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39664->77.90.31.11:4553: use of closed network connection Aug 9 12:12:17.518463446 (18h46m39.477560683s) [noise=true , address=45.91.116.8:4045 , piece_id=BFBMXTDDWR6WBCQETLUJLDHK7BFJCETZLMJXBZP2X5UONJ6QMIGA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38234->45.91.116.8:4045: use of closed network connection Aug 9 12:12:17.518531998 (18h46m39.477629233s) [noise=true , address=45.91.116.7:4009 , piece_id=CYN6K5GNBJ7FGJ5YSS2QEA2QCRCS75TX54UFCNO34BAFRFRR3L3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40272->45.91.116.7:4009: use of closed network connection Aug 9 12:12:17.518554941 (18h46m39.477652179s) [noise=true , address=45.91.116.6:4973 , piece_id=I46N2RJKJSUJ36ZSY6OHHP6CZTJW7YH5TNA5GNRBB2XPD6BEHPYA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.518659145 (18h46m39.47775638s) [noise=true , address=45.91.116.14:4190 , piece_id=SJNZOESF4WWSMSWL4HSJSRHINFT6J5Z72PF5UFLUN5FM2NMAW7WQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55832->45.91.116.14:4190: use of closed network connection Aug 9 12:12:17.518780738 (18h46m39.47787797s) [noise=true , address=45.91.116.10:4083 , piece_id=JSPQDTGH7F4LINARGOGCZMFMRQM5MGLBPGQ6PZR465YAD5TAVPHA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38462->45.91.116.10:4083: use of closed network connection Aug 9 12:12:17.518864073 (18h46m39.477961304s) [noise=true , address=45.91.116.5:4297 , piece_id=CXEURKP3UYNT3KNKO2AM6RJDHPPBLSCSJAGCSANYHNQSFIKC7ACA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35920->45.91.116.5:4297: use of closed network connection Aug 9 12:12:17.518927033 (18h46m39.478024279s) [noise=true , address=77.90.31.16:4734 , piece_id=JMK4ZMQNFHK6Z6LE2F7NBGY52DSWK56E6JYQWR2257ALZW5FNUNQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58878->77.90.31.16:4734: use of closed network connection Aug 9 12:12:17.518981502 (18h46m39.478078739s) [noise=true , address=45.91.116.13:4153 , piece_id=PHZLKTMCCFZB54CLAN5RRUZ7EJJSMMA22SENENPWMNWADZBYO4AQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57274->45.91.116.13:4153: use of closed network connection Aug 9 12:12:17.519043419 (18h46m39.478140663s) [noise=true , address=77.90.31.15:4699 , piece_id=Y77PB2XLOI3GEITMYDZC4IZBJWYUTWDTHKUIM72O3AEQ7GABKOIA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519061815 (18h46m39.47815905s) [noise=true , address=45.91.116.9:4261 , piece_id=NY56CXCKPXYGC7CC6KMPHSNBX47YDWJN4ZP242JZBLBMVJCBECCQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41022->45.91.116.9:4261: use of closed network connection Aug 9 12:12:17.519155895 (18h46m39.478253131s) [noise=true , address=77.90.31.14:4663 , piece_id=RXM5UCCLPM7TQ2FX6SUX3FMCFVYF54XAURZATJD64D4E5KJTBZEA, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45626->77.90.31.14:4663: use of closed network connection Aug 9 12:12:17.519168763 (18h46m39.47826602s) [noise=true , address=77.90.31.19:4842 , piece_id=CKENCKYR6KONSJERTTAC2NGWILJOZCVOJDZJVLGZA77QGZ3WTJ3A, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 12:12:17.519193619 (18h46m39.478290854s) [noise=true , address=77.90.31.17:4771 , piece_id=CEK3D2BDWFH266BUK64KLPV426PUG4C6WVYCVXJTL6AJGHWDC7UQ, seg_pos={0 0}, upload=19553]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34274->77.90.31.17:4771: use of closed network connection Aug 9 12:12:17.519239090 (18h46m39.478336446s) [seg_pos={0 0}, upload=19553]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 9 12:12:17.623909256 (18h46m39.583006502s) [upload=19553]: Done uploading segments... Aug 9 12:12:17.623941281 (18h46m39.583038513s) [upload=19553]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 30.107491027s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 848.526223ms FAIL (2025-08-09 12:11:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 848.55718ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.427825063s FAIL (2025-08-09 12:11:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 2.427853957s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.269695828s FAIL (2025-08-09 12:11:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4226018 (18h11m29.273323277s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314415633 (18h11m29.273512881s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314726594 (18h11m29.273823839s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315035831 (18h11m29.274133076s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315361473 (18h11m29.274458723s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315801774 (18h11m29.274899021s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316102423 (18h11m29.27519967s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316378502 (18h11m29.275475752s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316709100 (18h11m29.275806353s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317222442 (18h11m29.276319697s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317598381 (18h11m29.276695629s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.318004669 (18h11m29.277101918s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.949397504 (18h11m29.90849476s) [upload=18948]: Next returned nil segment Aug 9 11:37:07.949420624 (18h11m29.908517904s) [upload=18948]: Waiting for error group managing segments... Aug 9 11:37:08.089551930 (18h11m30.048649175s) [noise=true , address=104.36.82.6:4874 , piece_id=Z6LRH3WR3QWODHLC2OP3JOQXABIIFASZUK3SS7N33KPN2OKQXDPA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.092412944 (18h11m30.051510189s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.099385074 (18h11m30.058482323s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.180090657 (18h11m30.139187895s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:24.004796612 (18h11m45.963893849s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:34.496190916 (18h11m56.455288156s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:36.097756932 (18h11m58.056854182s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.097973210 (18h11m58.057070448s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.098161794 (18h11m58.05725904s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection Aug 9 11:37:36.098207799 (18h11m58.057305037s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection Aug 9 11:37:36.098245218 (18h11m58.057342454s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection Aug 9 11:37:36.098287535 (18h11m58.057384779s) [noise=true , address=45.91.116.12:4226 , piece_id=KYCTUVJZH2QWVDFE35UZWPSMZ2ZZ2JKQDJCKEM7VEZN6OWEWVTZQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection Aug 9 11:37:36.098318065 (18h11m58.057415301s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection Aug 9 11:37:36.098362497 (18h11m58.057459737s) [noise=true , address=77.90.31.11:4553 , piece_id=IMSNXEECWHIASEQYZRASRJMT2PJYXDJ65OFBIEFUABVYLIXCKUYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection Aug 9 11:37:36.098370733 (18h11m58.057467969s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection Aug 9 11:37:36.098415055 (18h11m58.057512295s) [noise=true , address=45.91.116.12:4225 , piece_id=7TIJNRDL75TBLIMWKWOK7WPTIQIJ7XCH6R6YBDG45NET7UW6G6LQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection Aug 9 11:37:36.098439035 (18h11m58.057536289s) [seg_pos={0 0}, upload=18948]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:36.327886327 (18h11m58.286983577s) [upload=18948]: Done uploading segments... Aug 9 11:37:36.327920391 (18h11m58.287017643s) [upload=18948]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.269722565s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 845.487668ms FAIL (2025-08-09 12:10:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 845.520704ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.485198326s FAIL (2025-08-09 12:10:51 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 2.485240812s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.300288665s FAIL (2025-08-09 12:10:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4226018 (18h11m29.273323277s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314415633 (18h11m29.273512881s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314726594 (18h11m29.273823839s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315035831 (18h11m29.274133076s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315361473 (18h11m29.274458723s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315801774 (18h11m29.274899021s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316102423 (18h11m29.27519967s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316378502 (18h11m29.275475752s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316709100 (18h11m29.275806353s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317222442 (18h11m29.276319697s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317598381 (18h11m29.276695629s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.318004669 (18h11m29.277101918s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.949397504 (18h11m29.90849476s) [upload=18948]: Next returned nil segment Aug 9 11:37:07.949420624 (18h11m29.908517904s) [upload=18948]: Waiting for error group managing segments... Aug 9 11:37:08.089551930 (18h11m30.048649175s) [noise=true , address=104.36.82.6:4874 , piece_id=Z6LRH3WR3QWODHLC2OP3JOQXABIIFASZUK3SS7N33KPN2OKQXDPA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.092412944 (18h11m30.051510189s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.099385074 (18h11m30.058482323s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.180090657 (18h11m30.139187895s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:24.004796612 (18h11m45.963893849s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:34.496190916 (18h11m56.455288156s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:36.097756932 (18h11m58.056854182s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.097973210 (18h11m58.057070448s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.098161794 (18h11m58.05725904s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection Aug 9 11:37:36.098207799 (18h11m58.057305037s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection Aug 9 11:37:36.098245218 (18h11m58.057342454s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection Aug 9 11:37:36.098287535 (18h11m58.057384779s) [noise=true , address=45.91.116.12:4226 , piece_id=KYCTUVJZH2QWVDFE35UZWPSMZ2ZZ2JKQDJCKEM7VEZN6OWEWVTZQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection Aug 9 11:37:36.098318065 (18h11m58.057415301s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection Aug 9 11:37:36.098362497 (18h11m58.057459737s) [noise=true , address=77.90.31.11:4553 , piece_id=IMSNXEECWHIASEQYZRASRJMT2PJYXDJ65OFBIEFUABVYLIXCKUYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection Aug 9 11:37:36.098370733 (18h11m58.057467969s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection Aug 9 11:37:36.098415055 (18h11m58.057512295s) [noise=true , address=45.91.116.12:4225 , piece_id=7TIJNRDL75TBLIMWKWOK7WPTIQIJ7XCH6R6YBDG45NET7UW6G6LQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection Aug 9 11:37:36.098439035 (18h11m58.057536289s) [seg_pos={0 0}, upload=18948]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:36.327886327 (18h11m58.286983577s) [upload=18948]: Done uploading segments... Aug 9 11:37:36.327920391 (18h11m58.287017643s) [upload=18948]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.300317703s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 914.73803ms FAIL (2025-08-09 12:10:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 914.770234ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)