Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [..XXX.....XX.X..X.XX..X..X.X.............X........X.X.......] duration: 2.606821123s FAIL (2025-09-22 04:42:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.606853911s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [....................XX..X..X......X..............X.X........] duration: 2.75591245s FAIL (2025-09-22 04:42:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.755948471s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....................XX..X..X......X..............X.X.......] duration: 3.925513214s FAIL (2025-09-22 04:41:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.92652139s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [...XXX.....XX.X..X.XX..X..X.X.............X........X.X......] duration: 2.516562416s FAIL (2025-09-22 04:41:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.516594491s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [......................XX..X..X......X..............X.X......] duration: 2.698427068s FAIL (2025-09-22 04:40:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.698451826s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.......................XX..X..X......X..............X.X.....] duration: 3.190237076s FAIL (2025-09-22 04:40:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.190274846s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [....XXX.....XX.X..X.XX..X..X.X.............X........X.X.....] duration: 2.494454715s FAIL (2025-09-22 04:40:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.494497759s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [........................XX..X..X......X..............X.X....] duration: 2.819660327s FAIL (2025-09-22 04:40:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.819798074s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.........................XX..X..X......X..............X.X...] duration: 2.937270103s FAIL (2025-09-22 04:40:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.937321685s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [X.........................XX..X..X......X..............X.X..] duration: 2.795634179s FAIL (2025-09-22 04:39:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.795712631s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.....XXX.....XX.X..X.XX..X..X.X.............X........X.X....] duration: 2.294778644s FAIL (2025-09-22 04:39:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.294809395s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [XX.........................XX..X..X......X..............X.X.] duration: 2.62855706s FAIL (2025-09-22 04:38:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.628591481s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [......XXX.....XX.X..X.XX..X..X.X.............X........X.X...] duration: 2.423073535s FAIL (2025-09-22 04:38:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.42316553s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [X......XXX.....XX.X..X.XX..X..X.X.............X........X.X..] duration: 2.535647694s FAIL (2025-09-22 04:38:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.535894085s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [XXX.........................XX..X..X......X..............X.X] duration: 30.001310752s FAIL (2025-09-22 04:38:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001344955s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [.X......XXX.....XX.X..X.XX..X..X.X.............X........X.X.] duration: 2.694522246s FAIL (2025-09-22 04:38:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.694553749s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [..X......XXX.....XX.X..X.XX..X..X.X.............X........X.X] duration: 30.108728943s FAIL (2025-09-22 04:38:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.118134428s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [X..X......XXX.....XX.X..X.XX..X..X.X.............X........X.] duration: 2.392213071s FAIL (2025-09-22 04:37:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 04:36:39.543145911 (566h29m0.035578357s) [upload=498343]: Starting upload Sep 22 04:36:39.543212693 (566h29m0.035645142s) [upload=498343]: Done starting upload Sep 22 04:36:39.543379867 (566h29m0.035812322s) [upload=498343]: Uploading segments... Sep 22 04:36:39.543548896 (566h29m0.035981326s) [upload=498343]: Got next segment. Inline: false Sep 22 04:36:39.543614991 (566h29m0.036047434s) [upload=498343]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:39.543621053 (566h29m0.036053507s) [upload=498343]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:39.694955727 (566h29m0.18738818s) [seg_pos={0 0}, upload=498343]: Begin upload segment... Sep 22 04:36:39.695042947 (566h29m0.187475391s) [seg_pos={0 0}, upload=498343]: Done begin upload segment. Sep 22 04:36:39.695057000 (566h29m0.187489447s) [noise=true , address=45.91.116.11:4118 , piece_id=SNQJFMYMVVOVA4QSTJHJFI2BXGDYOSQU2LND7CBISB6GFQVG2CGQ, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.695101112 (566h29m0.187533565s) [noise=true , address=145.79.227.2:4113 , piece_id=7JDYWNKXQJIJTL2NQ7O64KQNWOWKWYWDEINXOPR6IWHH37GSTF3Q, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.695506341 (566h29m0.187938786s) [noise=true , address=45.91.116.5:4298 , piece_id=57XKPJJJYDLAX2G2C3QFELA74CYWXDVXHEQE3RLYIFBXAI5IM6XQ, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.695594475 (566h29m0.188026929s) [noise=true , address=145.79.227.15:4876 , piece_id=2OG77JDLJZBNB3AYMSGDJBKB4VHKDJALBPN4CVKOFCLLMQUTKNKA, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.696030993 (566h29m0.188463446s) [noise=true , address=145.79.227.14:4840 , piece_id=5LL2LZSPZPPN5SC55SJWYZ2ZM24AWQJSX74J7K3DZW4TSN5TJWDA, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.696436450 (566h29m0.188868905s) [noise=true , address=145.79.227.16:4910 , piece_id=IPTZTKGTBXDLWQM2PMJRW37I6MELXYOEVR66V7W7O3JLOCORWCUQ, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.696834182 (566h29m0.189266636s) [noise=true , address=77.90.31.14:4663 , piece_id=Z2TZH5LP734V7LBUUL2LL6R3WE6ETYBM737WSV4P5JOWV2S5UZ2A, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.697219184 (566h29m0.189651636s) [noise=true , address=145.79.227.5:4223 , piece_id=AAWAM54YVDAJDAQ4Y6XKVTTGN6TWW75TML2VM7VH27DOBBT6V3XA, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.697386127 (566h29m0.189818569s) [noise=true , address=45.91.116.13:4154 , piece_id=NWJNJHOQCQHGE5IFVKZ5QOQIUYL3H5THRIYJBUI2CJJARKPFJPYA, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.697792628 (566h29m0.190225077s) [noise=true , address=45.91.116.13:4155 , piece_id=IHHT6BMQW2LGLCJWU4ODCI5WMFIMMJFG3TITLWHPFI5XNSFQOORQ, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.698130843 (566h29m0.190563291s) [noise=true , address=77.90.31.19:4843 , piece_id=HII24ZXJ7TPNN5K4QZ4YZSG4PHAKLP4LZWSKBKYFDLNNZH7EVM4A, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.698590061 (566h29m0.191022517s) [noise=true , address=77.90.31.10:4520 , piece_id=ATUHXIIS2AGYIQP4P4547GNFUUB3PNYOSBZHK2WWV6WBZBSQLMRQ, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.698865538 (566h29m0.191297987s) [noise=true , address=45.91.116.11:4117 , piece_id=46MKMT7KQMR7O462DU645KBPCHIESY3RX2WMA4LZTVRAOYZ4MSEA, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.699322383 (566h29m0.191754835s) [noise=true , address=45.91.116.8:4045 , piece_id=T32XTUBFSRN37CYEF7ASWGEYBKSIFMACN7ISLRMYKE3MCVF736HA, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.699571641 (566h29m0.192004082s) [noise=true , address=77.90.31.11:4555 , piece_id=FYO4OIN3MGKE725GODRL2RTRPPCXFLFGIYGY24KXR3OWRMWD4H4Q, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.699940912 (566h29m0.192373361s) [noise=true , address=45.91.116.12:4226 , piece_id=MAY53XFOZ7NGTFAJVIEGRPX5GPEYL6CRXCB2Q2BCHBGVVQBLOZ4A, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:40.317595250 (566h29m0.810027686s) [upload=498343]: Next returned nil segment Sep 22 04:36:40.317614233 (566h29m0.810046681s) [upload=498343]: Waiting for error group managing segments... Sep 22 04:36:40.445336930 (566h29m0.937769375s) [noise=true , address=45.91.116.11:4118 , piece_id=SNQJFMYMVVOVA4QSTJHJFI2BXGDYOSQU2LND7CBISB6GFQVG2CGQ, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:40.447247117 (566h29m0.939679566s) [noise=true , address=45.91.116.11:4117 , piece_id=46MKMT7KQMR7O462DU645KBPCHIESY3RX2WMA4LZTVRAOYZ4MSEA, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:40.447556106 (566h29m0.939988539s) [noise=true , address=77.90.31.14:4663 , piece_id=Z2TZH5LP734V7LBUUL2LL6R3WE6ETYBM737WSV4P5JOWV2S5UZ2A, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:40.448352995 (566h29m0.940785443s) [noise=true , address=45.91.116.5:4298 , piece_id=57XKPJJJYDLAX2G2C3QFELA74CYWXDVXHEQE3RLYIFBXAI5IM6XQ, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:40.448723250 (566h29m0.941155693s) [noise=true , address=77.90.31.19:4843 , piece_id=HII24ZXJ7TPNN5K4QZ4YZSG4PHAKLP4LZWSKBKYFDLNNZH7EVM4A, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:40.449245778 (566h29m0.941678224s) [noise=true , address=77.90.31.11:4555 , piece_id=FYO4OIN3MGKE725GODRL2RTRPPCXFLFGIYGY24KXR3OWRMWD4H4Q, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:40.450905277 (566h29m0.943337726s) [noise=true , address=77.90.31.10:4520 , piece_id=ATUHXIIS2AGYIQP4P4547GNFUUB3PNYOSBZHK2WWV6WBZBSQLMRQ, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:40.455302812 (566h29m0.947735241s) [noise=true , address=45.91.116.13:4154 , piece_id=NWJNJHOQCQHGE5IFVKZ5QOQIUYL3H5THRIYJBUI2CJJARKPFJPYA, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:40.457088087 (566h29m0.949520516s) [noise=true , address=45.91.116.13:4155 , piece_id=IHHT6BMQW2LGLCJWU4ODCI5WMFIMMJFG3TITLWHPFI5XNSFQOORQ, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:40.458032911 (566h29m0.95046534s) [noise=true , address=45.91.116.8:4045 , piece_id=T32XTUBFSRN37CYEF7ASWGEYBKSIFMACN7ISLRMYKE3MCVF736HA, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:40.460709402 (566h29m0.953141831s) [noise=true , address=45.91.116.12:4226 , piece_id=MAY53XFOZ7NGTFAJVIEGRPX5GPEYL6CRXCB2Q2BCHBGVVQBLOZ4A, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:42.521380846 (566h29m3.013813298s) [noise=true , address=145.79.227.16:4910 , piece_id=IPTZTKGTBXDLWQM2PMJRW37I6MELXYOEVR66V7W7O3JLOCORWCUQ, seg_pos={0 0}, upload=498343]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:37:09.273403290 (566h29m29.765835749s) [noise=true , address=145.79.227.5:4223 , piece_id=AAWAM54YVDAJDAQ4Y6XKVTTGN6TWW75TML2VM7VH27DOBBT6V3XA, seg_pos={0 0}, upload=498343]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: rpc: dial tcp 145.79.227.5:4223: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:37:09.273505934 (566h29m29.765938383s) [noise=true , address=145.79.227.2:4113 , piece_id=7JDYWNKXQJIJTL2NQ7O64KQNWOWKWYWDEINXOPR6IWHH37GSTF3Q, seg_pos={0 0}, upload=498343]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:37:09.273569833 (566h29m29.766002289s) [noise=true , address=145.79.227.14:4840 , piece_id=5LL2LZSPZPPN5SC55SJWYZ2ZM24AWQJSX74J7K3DZW4TSN5TJWDA, seg_pos={0 0}, upload=498343]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:37:09.273621910 (566h29m29.766054367s) [noise=true , address=145.79.227.15:4876 , piece_id=2OG77JDLJZBNB3AYMSGDJBKB4VHKDJALBPN4CVKOFCLLMQUTKNKA, seg_pos={0 0}, upload=498343]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:37:09.273641448 (566h29m29.766073909s) [seg_pos={0 0}, upload=498343]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 04:37:09.390860665 (566h29m29.8832931s) [upload=498343]: Done uploading segments... Sep 22 04:37:09.390884990 (566h29m29.883317421s) [upload=498343]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.392240104s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [XX..X......XXX.....XX.X..X.XX..X..X.X.............X........X] duration: 30.155217583s FAIL (2025-09-22 04:37:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 04:36:39.543145911 (566h29m0.035578357s) [upload=498343]: Starting upload Sep 22 04:36:39.543212693 (566h29m0.035645142s) [upload=498343]: Done starting upload Sep 22 04:36:39.543379867 (566h29m0.035812322s) [upload=498343]: Uploading segments... Sep 22 04:36:39.543548896 (566h29m0.035981326s) [upload=498343]: Got next segment. Inline: false Sep 22 04:36:39.543614991 (566h29m0.036047434s) [upload=498343]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:39.543621053 (566h29m0.036053507s) [upload=498343]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:39.694955727 (566h29m0.18738818s) [seg_pos={0 0}, upload=498343]: Begin upload segment... Sep 22 04:36:39.695042947 (566h29m0.187475391s) [seg_pos={0 0}, upload=498343]: Done begin upload segment. Sep 22 04:36:39.695057000 (566h29m0.187489447s) [noise=true , address=45.91.116.11:4118 , piece_id=SNQJFMYMVVOVA4QSTJHJFI2BXGDYOSQU2LND7CBISB6GFQVG2CGQ, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.695101112 (566h29m0.187533565s) [noise=true , address=145.79.227.2:4113 , piece_id=7JDYWNKXQJIJTL2NQ7O64KQNWOWKWYWDEINXOPR6IWHH37GSTF3Q, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.695506341 (566h29m0.187938786s) [noise=true , address=45.91.116.5:4298 , piece_id=57XKPJJJYDLAX2G2C3QFELA74CYWXDVXHEQE3RLYIFBXAI5IM6XQ, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.695594475 (566h29m0.188026929s) [noise=true , address=145.79.227.15:4876 , piece_id=2OG77JDLJZBNB3AYMSGDJBKB4VHKDJALBPN4CVKOFCLLMQUTKNKA, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.696030993 (566h29m0.188463446s) [noise=true , address=145.79.227.14:4840 , piece_id=5LL2LZSPZPPN5SC55SJWYZ2ZM24AWQJSX74J7K3DZW4TSN5TJWDA, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.696436450 (566h29m0.188868905s) [noise=true , address=145.79.227.16:4910 , piece_id=IPTZTKGTBXDLWQM2PMJRW37I6MELXYOEVR66V7W7O3JLOCORWCUQ, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.696834182 (566h29m0.189266636s) [noise=true , address=77.90.31.14:4663 , piece_id=Z2TZH5LP734V7LBUUL2LL6R3WE6ETYBM737WSV4P5JOWV2S5UZ2A, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.697219184 (566h29m0.189651636s) [noise=true , address=145.79.227.5:4223 , piece_id=AAWAM54YVDAJDAQ4Y6XKVTTGN6TWW75TML2VM7VH27DOBBT6V3XA, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.697386127 (566h29m0.189818569s) [noise=true , address=45.91.116.13:4154 , piece_id=NWJNJHOQCQHGE5IFVKZ5QOQIUYL3H5THRIYJBUI2CJJARKPFJPYA, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.697792628 (566h29m0.190225077s) [noise=true , address=45.91.116.13:4155 , piece_id=IHHT6BMQW2LGLCJWU4ODCI5WMFIMMJFG3TITLWHPFI5XNSFQOORQ, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.698130843 (566h29m0.190563291s) [noise=true , address=77.90.31.19:4843 , piece_id=HII24ZXJ7TPNN5K4QZ4YZSG4PHAKLP4LZWSKBKYFDLNNZH7EVM4A, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.698590061 (566h29m0.191022517s) [noise=true , address=77.90.31.10:4520 , piece_id=ATUHXIIS2AGYIQP4P4547GNFUUB3PNYOSBZHK2WWV6WBZBSQLMRQ, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.698865538 (566h29m0.191297987s) [noise=true , address=45.91.116.11:4117 , piece_id=46MKMT7KQMR7O462DU645KBPCHIESY3RX2WMA4LZTVRAOYZ4MSEA, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.699322383 (566h29m0.191754835s) [noise=true , address=45.91.116.8:4045 , piece_id=T32XTUBFSRN37CYEF7ASWGEYBKSIFMACN7ISLRMYKE3MCVF736HA, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.699571641 (566h29m0.192004082s) [noise=true , address=77.90.31.11:4555 , piece_id=FYO4OIN3MGKE725GODRL2RTRPPCXFLFGIYGY24KXR3OWRMWD4H4Q, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:39.699940912 (566h29m0.192373361s) [noise=true , address=45.91.116.12:4226 , piece_id=MAY53XFOZ7NGTFAJVIEGRPX5GPEYL6CRXCB2Q2BCHBGVVQBLOZ4A, seg_pos={0 0}, upload=498343]: Uploading piece... Sep 22 04:36:40.317595250 (566h29m0.810027686s) [upload=498343]: Next returned nil segment Sep 22 04:36:40.317614233 (566h29m0.810046681s) [upload=498343]: Waiting for error group managing segments... Sep 22 04:36:40.445336930 (566h29m0.937769375s) [noise=true , address=45.91.116.11:4118 , piece_id=SNQJFMYMVVOVA4QSTJHJFI2BXGDYOSQU2LND7CBISB6GFQVG2CGQ, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:40.447247117 (566h29m0.939679566s) [noise=true , address=45.91.116.11:4117 , piece_id=46MKMT7KQMR7O462DU645KBPCHIESY3RX2WMA4LZTVRAOYZ4MSEA, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:40.447556106 (566h29m0.939988539s) [noise=true , address=77.90.31.14:4663 , piece_id=Z2TZH5LP734V7LBUUL2LL6R3WE6ETYBM737WSV4P5JOWV2S5UZ2A, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:40.448352995 (566h29m0.940785443s) [noise=true , address=45.91.116.5:4298 , piece_id=57XKPJJJYDLAX2G2C3QFELA74CYWXDVXHEQE3RLYIFBXAI5IM6XQ, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:40.448723250 (566h29m0.941155693s) [noise=true , address=77.90.31.19:4843 , piece_id=HII24ZXJ7TPNN5K4QZ4YZSG4PHAKLP4LZWSKBKYFDLNNZH7EVM4A, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:40.449245778 (566h29m0.941678224s) [noise=true , address=77.90.31.11:4555 , piece_id=FYO4OIN3MGKE725GODRL2RTRPPCXFLFGIYGY24KXR3OWRMWD4H4Q, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:40.450905277 (566h29m0.943337726s) [noise=true , address=77.90.31.10:4520 , piece_id=ATUHXIIS2AGYIQP4P4547GNFUUB3PNYOSBZHK2WWV6WBZBSQLMRQ, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:40.455302812 (566h29m0.947735241s) [noise=true , address=45.91.116.13:4154 , piece_id=NWJNJHOQCQHGE5IFVKZ5QOQIUYL3H5THRIYJBUI2CJJARKPFJPYA, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:40.457088087 (566h29m0.949520516s) [noise=true , address=45.91.116.13:4155 , piece_id=IHHT6BMQW2LGLCJWU4ODCI5WMFIMMJFG3TITLWHPFI5XNSFQOORQ, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:40.458032911 (566h29m0.95046534s) [noise=true , address=45.91.116.8:4045 , piece_id=T32XTUBFSRN37CYEF7ASWGEYBKSIFMACN7ISLRMYKE3MCVF736HA, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:40.460709402 (566h29m0.953141831s) [noise=true , address=45.91.116.12:4226 , piece_id=MAY53XFOZ7NGTFAJVIEGRPX5GPEYL6CRXCB2Q2BCHBGVVQBLOZ4A, seg_pos={0 0}, upload=498343]: Done uploading piece. err: <nil> Sep 22 04:36:42.521380846 (566h29m3.013813298s) [noise=true , address=145.79.227.16:4910 , piece_id=IPTZTKGTBXDLWQM2PMJRW37I6MELXYOEVR66V7W7O3JLOCORWCUQ, seg_pos={0 0}, upload=498343]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:37:09.273403290 (566h29m29.765835749s) [noise=true , address=145.79.227.5:4223 , piece_id=AAWAM54YVDAJDAQ4Y6XKVTTGN6TWW75TML2VM7VH27DOBBT6V3XA, seg_pos={0 0}, upload=498343]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: rpc: dial tcp 145.79.227.5:4223: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:37:09.273505934 (566h29m29.765938383s) [noise=true , address=145.79.227.2:4113 , piece_id=7JDYWNKXQJIJTL2NQ7O64KQNWOWKWYWDEINXOPR6IWHH37GSTF3Q, seg_pos={0 0}, upload=498343]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:37:09.273569833 (566h29m29.766002289s) [noise=true , address=145.79.227.14:4840 , piece_id=5LL2LZSPZPPN5SC55SJWYZ2ZM24AWQJSX74J7K3DZW4TSN5TJWDA, seg_pos={0 0}, upload=498343]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:37:09.273621910 (566h29m29.766054367s) [noise=true , address=145.79.227.15:4876 , piece_id=2OG77JDLJZBNB3AYMSGDJBKB4VHKDJALBPN4CVKOFCLLMQUTKNKA, seg_pos={0 0}, upload=498343]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:37:09.273641448 (566h29m29.766073909s) [seg_pos={0 0}, upload=498343]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 04:37:09.390860665 (566h29m29.8832931s) [upload=498343]: Done uploading segments... Sep 22 04:37:09.390884990 (566h29m29.883317421s) [upload=498343]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.155243977s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [XXXX.........................XX..X..X......X..............X.] duration: 2.848069325s FAIL (2025-09-22 04:36:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.848093571s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [.XX..X......XXX.....XX.X..X.XX..X..X.X.............X........] duration: 2.422182514s FAIL (2025-09-22 04:36:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:31:24.920143949 (566h23m45.41257638s) [upload=498269]: Starting upload Sep 22 04:31:24.920215948 (566h23m45.412648393s) [upload=498269]: Done starting upload Sep 22 04:31:24.920362595 (566h23m45.412795038s) [upload=498269]: Uploading segments... Sep 22 04:31:24.920492684 (566h23m45.412925131s) [upload=498269]: Got next segment. Inline: false Sep 22 04:31:24.920572454 (566h23m45.41300489s) [upload=498269]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:31:24.920578942 (566h23m45.413011385s) [upload=498269]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:31:25.045745592 (566h23m45.53817805s) [seg_pos={0 0}, upload=498269]: Begin upload segment... Sep 22 04:31:25.045861133 (566h23m45.538293585s) [seg_pos={0 0}, upload=498269]: Done begin upload segment. Sep 22 04:31:25.045945099 (566h23m45.53837755s) [noise=true , address=45.91.116.9:4261 , piece_id=CEGMAKDYVS3WEZVJLWZ7PLPIFF5HGU72XR3RYN5KFI6JYY6773MQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.045969575 (566h23m45.538402017s) [noise=true , address=145.79.227.2:4115 , piece_id=4BO6TSTGW7MMJTLXNQFU55C7I2OBGPJZWQKW4R5U2SPAWQS6IHPA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046521915 (566h23m45.538954365s) [noise=true , address=45.91.116.13:4155 , piece_id=7S37SPYHOEXLMC6QHLGZLH7KRXD7SCV3U4LQ62OBO2GOEBMY4DXA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046575700 (566h23m45.539008155s) [noise=true , address=145.79.227.18:4005 , piece_id=IMNWS75R4MTBLDUVYR6TCOWZLB5U3763R56HKKO3LSKGJVPSIGNQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046904954 (566h23m45.539337399s) [noise=true , address=45.91.116.8:4045 , piece_id=WQBNA3NDZBQ6NAMLMB4IKGOZIH4SHZACNOSMRQBUJXMSO4AKESVA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047049339 (566h23m45.53948179s) [noise=true , address=45.91.116.11:4117 , piece_id=2ZNQUGE6R2QOZJCRA4KWT2Y7IKZ5ZPQWTKNTSMK3WOZ4KUN5KQZQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047469004 (566h23m45.539901457s) [noise=true , address=77.90.31.19:4841 , piece_id=ZFNAUEMLT6VWPVIP53EKINHKNWC4H7UNJSTL2OYUS6WPFAUOPS5A, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047957742 (566h23m45.540390184s) [noise=true , address=77.90.31.12:4589 , piece_id=J2VRYHV43YRSOJ2X7NXRSO35O4JNBJELON27GFGJBSQG3L7DIKUA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048390500 (566h23m45.540822989s) [noise=true , address=145.79.227.17:4970 , piece_id=A5S2OJM5VUCSEZAJFOWHQK2XUKXUTT5VVSQLKK6ZC62ARU6WZSPQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048619410 (566h23m45.541051859s) [noise=true , address=77.90.31.15:4697 , piece_id=NBBDIDC64NAZKUYUGLHV5DAHDZKHHDHDWS3OBQG3VF5GOUHZVPWA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048962659 (566h23m45.541395101s) [noise=true , address=45.91.116.7:4010 , piece_id=OWKSKTCLX3JFSDOCEP37IHTAO32TICL7F7YMSK6KIH47ZRGJIT5Q, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049294858 (566h23m45.541727303s) [noise=true , address=77.90.31.16:4734 , piece_id=6PPHPDJRMRKNEYHMDDZVCLICO5KUXMZAFBZ7EF6XVMYKGSPTVDSQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049592400 (566h23m45.542024845s) [noise=true , address=77.90.31.10:4520 , piece_id=AGNOJKORSH4CEZUJ2P6CFFHVNWK6IB2S5GXUSXGU4ALMPZJCFTQQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049939474 (566h23m45.542371921s) [noise=true , address=145.79.227.17:4971 , piece_id=RWVET42YTHRDPI7D6GRSYUK7EIDRSWCZ5VNDIKCA4VRMGUCVBAFQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.050366023 (566h23m45.542798478s) [noise=true , address=45.91.116.14:4190 , piece_id=TS3M625BQNTBC4YBH3GOVRJ323CBRV34K3CAJMX7FNSQBYWSSOYQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.050850463 (566h23m45.543282914s) [noise=true , address=77.90.31.18:4805 , piece_id=F5IQFMGK7T5IGFALDPPMC4QBKMFUQ2OG7622QA2YJBUQDWBS4CVA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.667633076 (566h23m46.160065509s) [upload=498269]: Next returned nil segment Sep 22 04:31:25.667664989 (566h23m46.160097432s) [upload=498269]: Waiting for error group managing segments... Sep 22 04:31:25.803338183 (566h23m46.295770615s) [noise=true , address=45.91.116.8:4045 , piece_id=WQBNA3NDZBQ6NAMLMB4IKGOZIH4SHZACNOSMRQBUJXMSO4AKESVA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.805064032 (566h23m46.297496465s) [noise=true , address=77.90.31.10:4520 , piece_id=AGNOJKORSH4CEZUJ2P6CFFHVNWK6IB2S5GXUSXGU4ALMPZJCFTQQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.806116220 (566h23m46.298548651s) [noise=true , address=45.91.116.7:4010 , piece_id=OWKSKTCLX3JFSDOCEP37IHTAO32TICL7F7YMSK6KIH47ZRGJIT5Q, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.806176194 (566h23m46.298608727s) [noise=true , address=45.91.116.9:4261 , piece_id=CEGMAKDYVS3WEZVJLWZ7PLPIFF5HGU72XR3RYN5KFI6JYY6773MQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.809766126 (566h23m46.30219857s) [noise=true , address=45.91.116.14:4190 , piece_id=TS3M625BQNTBC4YBH3GOVRJ323CBRV34K3CAJMX7FNSQBYWSSOYQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.810019259 (566h23m46.302451696s) [noise=true , address=45.91.116.11:4117 , piece_id=2ZNQUGE6R2QOZJCRA4KWT2Y7IKZ5ZPQWTKNTSMK3WOZ4KUN5KQZQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.810642598 (566h23m46.303075034s) [noise=true , address=77.90.31.12:4589 , piece_id=J2VRYHV43YRSOJ2X7NXRSO35O4JNBJELON27GFGJBSQG3L7DIKUA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.811792355 (566h23m46.304224799s) [noise=true , address=77.90.31.19:4841 , piece_id=ZFNAUEMLT6VWPVIP53EKINHKNWC4H7UNJSTL2OYUS6WPFAUOPS5A, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.811940311 (566h23m46.304372744s) [noise=true , address=45.91.116.13:4155 , piece_id=7S37SPYHOEXLMC6QHLGZLH7KRXD7SCV3U4LQ62OBO2GOEBMY4DXA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.812546498 (566h23m46.304978934s) [noise=true , address=77.90.31.18:4805 , piece_id=F5IQFMGK7T5IGFALDPPMC4QBKMFUQ2OG7622QA2YJBUQDWBS4CVA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.813702226 (566h23m46.306134661s) [noise=true , address=77.90.31.16:4734 , piece_id=6PPHPDJRMRKNEYHMDDZVCLICO5KUXMZAFBZ7EF6XVMYKGSPTVDSQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.814617534 (566h23m46.307049965s) [noise=true , address=77.90.31.15:4697 , piece_id=NBBDIDC64NAZKUYUGLHV5DAHDZKHHDHDWS3OBQG3VF5GOUHZVPWA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:41.942437754 (566h24m2.43487019s) [noise=true , address=145.79.227.18:4005 , piece_id=IMNWS75R4MTBLDUVYR6TCOWZLB5U3763R56HKKO3LSKGJVPSIGNQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:31:54.607167993 (566h24m15.099600436s) [noise=true , address=145.79.227.17:4970 , piece_id=A5S2OJM5VUCSEZAJFOWHQK2XUKXUTT5VVSQLKK6ZC62ARU6WZSPQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607533226 (566h24m15.099965694s) [noise=true , address=145.79.227.17:4971 , piece_id=RWVET42YTHRDPI7D6GRSYUK7EIDRSWCZ5VNDIKCA4VRMGUCVBAFQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607775799 (566h24m15.100208249s) [noise=true , address=145.79.227.2:4115 , piece_id=4BO6TSTGW7MMJTLXNQFU55C7I2OBGPJZWQKW4R5U2SPAWQS6IHPA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607801850 (566h24m15.100234304s) [seg_pos={0 0}, upload=498269]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:31:54.718884679 (566h24m15.211317115s) [upload=498269]: Done uploading segments... Sep 22 04:31:54.718934548 (566h24m15.211366981s) [upload=498269]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.422218967s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.XXXX.........................XX..X..X......X..............X] duration: 30.001918073s FAIL (2025-09-22 04:36:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001951107s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [..XX..X......XXX.....XX.X..X.XX..X..X.X.............X.......] duration: 2.509634398s FAIL (2025-09-22 04:35:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:31:24.920143949 (566h23m45.41257638s) [upload=498269]: Starting upload Sep 22 04:31:24.920215948 (566h23m45.412648393s) [upload=498269]: Done starting upload Sep 22 04:31:24.920362595 (566h23m45.412795038s) [upload=498269]: Uploading segments... Sep 22 04:31:24.920492684 (566h23m45.412925131s) [upload=498269]: Got next segment. Inline: false Sep 22 04:31:24.920572454 (566h23m45.41300489s) [upload=498269]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:31:24.920578942 (566h23m45.413011385s) [upload=498269]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:31:25.045745592 (566h23m45.53817805s) [seg_pos={0 0}, upload=498269]: Begin upload segment... Sep 22 04:31:25.045861133 (566h23m45.538293585s) [seg_pos={0 0}, upload=498269]: Done begin upload segment. Sep 22 04:31:25.045945099 (566h23m45.53837755s) [noise=true , address=45.91.116.9:4261 , piece_id=CEGMAKDYVS3WEZVJLWZ7PLPIFF5HGU72XR3RYN5KFI6JYY6773MQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.045969575 (566h23m45.538402017s) [noise=true , address=145.79.227.2:4115 , piece_id=4BO6TSTGW7MMJTLXNQFU55C7I2OBGPJZWQKW4R5U2SPAWQS6IHPA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046521915 (566h23m45.538954365s) [noise=true , address=45.91.116.13:4155 , piece_id=7S37SPYHOEXLMC6QHLGZLH7KRXD7SCV3U4LQ62OBO2GOEBMY4DXA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046575700 (566h23m45.539008155s) [noise=true , address=145.79.227.18:4005 , piece_id=IMNWS75R4MTBLDUVYR6TCOWZLB5U3763R56HKKO3LSKGJVPSIGNQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046904954 (566h23m45.539337399s) [noise=true , address=45.91.116.8:4045 , piece_id=WQBNA3NDZBQ6NAMLMB4IKGOZIH4SHZACNOSMRQBUJXMSO4AKESVA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047049339 (566h23m45.53948179s) [noise=true , address=45.91.116.11:4117 , piece_id=2ZNQUGE6R2QOZJCRA4KWT2Y7IKZ5ZPQWTKNTSMK3WOZ4KUN5KQZQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047469004 (566h23m45.539901457s) [noise=true , address=77.90.31.19:4841 , piece_id=ZFNAUEMLT6VWPVIP53EKINHKNWC4H7UNJSTL2OYUS6WPFAUOPS5A, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047957742 (566h23m45.540390184s) [noise=true , address=77.90.31.12:4589 , piece_id=J2VRYHV43YRSOJ2X7NXRSO35O4JNBJELON27GFGJBSQG3L7DIKUA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048390500 (566h23m45.540822989s) [noise=true , address=145.79.227.17:4970 , piece_id=A5S2OJM5VUCSEZAJFOWHQK2XUKXUTT5VVSQLKK6ZC62ARU6WZSPQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048619410 (566h23m45.541051859s) [noise=true , address=77.90.31.15:4697 , piece_id=NBBDIDC64NAZKUYUGLHV5DAHDZKHHDHDWS3OBQG3VF5GOUHZVPWA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048962659 (566h23m45.541395101s) [noise=true , address=45.91.116.7:4010 , piece_id=OWKSKTCLX3JFSDOCEP37IHTAO32TICL7F7YMSK6KIH47ZRGJIT5Q, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049294858 (566h23m45.541727303s) [noise=true , address=77.90.31.16:4734 , piece_id=6PPHPDJRMRKNEYHMDDZVCLICO5KUXMZAFBZ7EF6XVMYKGSPTVDSQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049592400 (566h23m45.542024845s) [noise=true , address=77.90.31.10:4520 , piece_id=AGNOJKORSH4CEZUJ2P6CFFHVNWK6IB2S5GXUSXGU4ALMPZJCFTQQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049939474 (566h23m45.542371921s) [noise=true , address=145.79.227.17:4971 , piece_id=RWVET42YTHRDPI7D6GRSYUK7EIDRSWCZ5VNDIKCA4VRMGUCVBAFQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.050366023 (566h23m45.542798478s) [noise=true , address=45.91.116.14:4190 , piece_id=TS3M625BQNTBC4YBH3GOVRJ323CBRV34K3CAJMX7FNSQBYWSSOYQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.050850463 (566h23m45.543282914s) [noise=true , address=77.90.31.18:4805 , piece_id=F5IQFMGK7T5IGFALDPPMC4QBKMFUQ2OG7622QA2YJBUQDWBS4CVA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.667633076 (566h23m46.160065509s) [upload=498269]: Next returned nil segment Sep 22 04:31:25.667664989 (566h23m46.160097432s) [upload=498269]: Waiting for error group managing segments... Sep 22 04:31:25.803338183 (566h23m46.295770615s) [noise=true , address=45.91.116.8:4045 , piece_id=WQBNA3NDZBQ6NAMLMB4IKGOZIH4SHZACNOSMRQBUJXMSO4AKESVA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.805064032 (566h23m46.297496465s) [noise=true , address=77.90.31.10:4520 , piece_id=AGNOJKORSH4CEZUJ2P6CFFHVNWK6IB2S5GXUSXGU4ALMPZJCFTQQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.806116220 (566h23m46.298548651s) [noise=true , address=45.91.116.7:4010 , piece_id=OWKSKTCLX3JFSDOCEP37IHTAO32TICL7F7YMSK6KIH47ZRGJIT5Q, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.806176194 (566h23m46.298608727s) [noise=true , address=45.91.116.9:4261 , piece_id=CEGMAKDYVS3WEZVJLWZ7PLPIFF5HGU72XR3RYN5KFI6JYY6773MQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.809766126 (566h23m46.30219857s) [noise=true , address=45.91.116.14:4190 , piece_id=TS3M625BQNTBC4YBH3GOVRJ323CBRV34K3CAJMX7FNSQBYWSSOYQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.810019259 (566h23m46.302451696s) [noise=true , address=45.91.116.11:4117 , piece_id=2ZNQUGE6R2QOZJCRA4KWT2Y7IKZ5ZPQWTKNTSMK3WOZ4KUN5KQZQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.810642598 (566h23m46.303075034s) [noise=true , address=77.90.31.12:4589 , piece_id=J2VRYHV43YRSOJ2X7NXRSO35O4JNBJELON27GFGJBSQG3L7DIKUA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.811792355 (566h23m46.304224799s) [noise=true , address=77.90.31.19:4841 , piece_id=ZFNAUEMLT6VWPVIP53EKINHKNWC4H7UNJSTL2OYUS6WPFAUOPS5A, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.811940311 (566h23m46.304372744s) [noise=true , address=45.91.116.13:4155 , piece_id=7S37SPYHOEXLMC6QHLGZLH7KRXD7SCV3U4LQ62OBO2GOEBMY4DXA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.812546498 (566h23m46.304978934s) [noise=true , address=77.90.31.18:4805 , piece_id=F5IQFMGK7T5IGFALDPPMC4QBKMFUQ2OG7622QA2YJBUQDWBS4CVA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.813702226 (566h23m46.306134661s) [noise=true , address=77.90.31.16:4734 , piece_id=6PPHPDJRMRKNEYHMDDZVCLICO5KUXMZAFBZ7EF6XVMYKGSPTVDSQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.814617534 (566h23m46.307049965s) [noise=true , address=77.90.31.15:4697 , piece_id=NBBDIDC64NAZKUYUGLHV5DAHDZKHHDHDWS3OBQG3VF5GOUHZVPWA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:41.942437754 (566h24m2.43487019s) [noise=true , address=145.79.227.18:4005 , piece_id=IMNWS75R4MTBLDUVYR6TCOWZLB5U3763R56HKKO3LSKGJVPSIGNQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:31:54.607167993 (566h24m15.099600436s) [noise=true , address=145.79.227.17:4970 , piece_id=A5S2OJM5VUCSEZAJFOWHQK2XUKXUTT5VVSQLKK6ZC62ARU6WZSPQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607533226 (566h24m15.099965694s) [noise=true , address=145.79.227.17:4971 , piece_id=RWVET42YTHRDPI7D6GRSYUK7EIDRSWCZ5VNDIKCA4VRMGUCVBAFQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607775799 (566h24m15.100208249s) [noise=true , address=145.79.227.2:4115 , piece_id=4BO6TSTGW7MMJTLXNQFU55C7I2OBGPJZWQKW4R5U2SPAWQS6IHPA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607801850 (566h24m15.100234304s) [seg_pos={0 0}, upload=498269]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:31:54.718884679 (566h24m15.211317115s) [upload=498269]: Done uploading segments... Sep 22 04:31:54.718934548 (566h24m15.211366981s) [upload=498269]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.509664562s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..XXXX.........................XX..X..X......X..............] duration: 2.699100877s FAIL (2025-09-22 04:35:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.699128608s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [X..XX..X......XXX.....XX.X..X.XX..X..X.X.............X......] duration: 2.400770553s FAIL (2025-09-22 04:34:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:31:24.920143949 (566h23m45.41257638s) [upload=498269]: Starting upload Sep 22 04:31:24.920215948 (566h23m45.412648393s) [upload=498269]: Done starting upload Sep 22 04:31:24.920362595 (566h23m45.412795038s) [upload=498269]: Uploading segments... Sep 22 04:31:24.920492684 (566h23m45.412925131s) [upload=498269]: Got next segment. Inline: false Sep 22 04:31:24.920572454 (566h23m45.41300489s) [upload=498269]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:31:24.920578942 (566h23m45.413011385s) [upload=498269]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:31:25.045745592 (566h23m45.53817805s) [seg_pos={0 0}, upload=498269]: Begin upload segment... Sep 22 04:31:25.045861133 (566h23m45.538293585s) [seg_pos={0 0}, upload=498269]: Done begin upload segment. Sep 22 04:31:25.045945099 (566h23m45.53837755s) [noise=true , address=45.91.116.9:4261 , piece_id=CEGMAKDYVS3WEZVJLWZ7PLPIFF5HGU72XR3RYN5KFI6JYY6773MQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.045969575 (566h23m45.538402017s) [noise=true , address=145.79.227.2:4115 , piece_id=4BO6TSTGW7MMJTLXNQFU55C7I2OBGPJZWQKW4R5U2SPAWQS6IHPA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046521915 (566h23m45.538954365s) [noise=true , address=45.91.116.13:4155 , piece_id=7S37SPYHOEXLMC6QHLGZLH7KRXD7SCV3U4LQ62OBO2GOEBMY4DXA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046575700 (566h23m45.539008155s) [noise=true , address=145.79.227.18:4005 , piece_id=IMNWS75R4MTBLDUVYR6TCOWZLB5U3763R56HKKO3LSKGJVPSIGNQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046904954 (566h23m45.539337399s) [noise=true , address=45.91.116.8:4045 , piece_id=WQBNA3NDZBQ6NAMLMB4IKGOZIH4SHZACNOSMRQBUJXMSO4AKESVA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047049339 (566h23m45.53948179s) [noise=true , address=45.91.116.11:4117 , piece_id=2ZNQUGE6R2QOZJCRA4KWT2Y7IKZ5ZPQWTKNTSMK3WOZ4KUN5KQZQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047469004 (566h23m45.539901457s) [noise=true , address=77.90.31.19:4841 , piece_id=ZFNAUEMLT6VWPVIP53EKINHKNWC4H7UNJSTL2OYUS6WPFAUOPS5A, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047957742 (566h23m45.540390184s) [noise=true , address=77.90.31.12:4589 , piece_id=J2VRYHV43YRSOJ2X7NXRSO35O4JNBJELON27GFGJBSQG3L7DIKUA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048390500 (566h23m45.540822989s) [noise=true , address=145.79.227.17:4970 , piece_id=A5S2OJM5VUCSEZAJFOWHQK2XUKXUTT5VVSQLKK6ZC62ARU6WZSPQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048619410 (566h23m45.541051859s) [noise=true , address=77.90.31.15:4697 , piece_id=NBBDIDC64NAZKUYUGLHV5DAHDZKHHDHDWS3OBQG3VF5GOUHZVPWA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048962659 (566h23m45.541395101s) [noise=true , address=45.91.116.7:4010 , piece_id=OWKSKTCLX3JFSDOCEP37IHTAO32TICL7F7YMSK6KIH47ZRGJIT5Q, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049294858 (566h23m45.541727303s) [noise=true , address=77.90.31.16:4734 , piece_id=6PPHPDJRMRKNEYHMDDZVCLICO5KUXMZAFBZ7EF6XVMYKGSPTVDSQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049592400 (566h23m45.542024845s) [noise=true , address=77.90.31.10:4520 , piece_id=AGNOJKORSH4CEZUJ2P6CFFHVNWK6IB2S5GXUSXGU4ALMPZJCFTQQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049939474 (566h23m45.542371921s) [noise=true , address=145.79.227.17:4971 , piece_id=RWVET42YTHRDPI7D6GRSYUK7EIDRSWCZ5VNDIKCA4VRMGUCVBAFQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.050366023 (566h23m45.542798478s) [noise=true , address=45.91.116.14:4190 , piece_id=TS3M625BQNTBC4YBH3GOVRJ323CBRV34K3CAJMX7FNSQBYWSSOYQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.050850463 (566h23m45.543282914s) [noise=true , address=77.90.31.18:4805 , piece_id=F5IQFMGK7T5IGFALDPPMC4QBKMFUQ2OG7622QA2YJBUQDWBS4CVA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.667633076 (566h23m46.160065509s) [upload=498269]: Next returned nil segment Sep 22 04:31:25.667664989 (566h23m46.160097432s) [upload=498269]: Waiting for error group managing segments... Sep 22 04:31:25.803338183 (566h23m46.295770615s) [noise=true , address=45.91.116.8:4045 , piece_id=WQBNA3NDZBQ6NAMLMB4IKGOZIH4SHZACNOSMRQBUJXMSO4AKESVA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.805064032 (566h23m46.297496465s) [noise=true , address=77.90.31.10:4520 , piece_id=AGNOJKORSH4CEZUJ2P6CFFHVNWK6IB2S5GXUSXGU4ALMPZJCFTQQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.806116220 (566h23m46.298548651s) [noise=true , address=45.91.116.7:4010 , piece_id=OWKSKTCLX3JFSDOCEP37IHTAO32TICL7F7YMSK6KIH47ZRGJIT5Q, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.806176194 (566h23m46.298608727s) [noise=true , address=45.91.116.9:4261 , piece_id=CEGMAKDYVS3WEZVJLWZ7PLPIFF5HGU72XR3RYN5KFI6JYY6773MQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.809766126 (566h23m46.30219857s) [noise=true , address=45.91.116.14:4190 , piece_id=TS3M625BQNTBC4YBH3GOVRJ323CBRV34K3CAJMX7FNSQBYWSSOYQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.810019259 (566h23m46.302451696s) [noise=true , address=45.91.116.11:4117 , piece_id=2ZNQUGE6R2QOZJCRA4KWT2Y7IKZ5ZPQWTKNTSMK3WOZ4KUN5KQZQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.810642598 (566h23m46.303075034s) [noise=true , address=77.90.31.12:4589 , piece_id=J2VRYHV43YRSOJ2X7NXRSO35O4JNBJELON27GFGJBSQG3L7DIKUA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.811792355 (566h23m46.304224799s) [noise=true , address=77.90.31.19:4841 , piece_id=ZFNAUEMLT6VWPVIP53EKINHKNWC4H7UNJSTL2OYUS6WPFAUOPS5A, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.811940311 (566h23m46.304372744s) [noise=true , address=45.91.116.13:4155 , piece_id=7S37SPYHOEXLMC6QHLGZLH7KRXD7SCV3U4LQ62OBO2GOEBMY4DXA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.812546498 (566h23m46.304978934s) [noise=true , address=77.90.31.18:4805 , piece_id=F5IQFMGK7T5IGFALDPPMC4QBKMFUQ2OG7622QA2YJBUQDWBS4CVA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.813702226 (566h23m46.306134661s) [noise=true , address=77.90.31.16:4734 , piece_id=6PPHPDJRMRKNEYHMDDZVCLICO5KUXMZAFBZ7EF6XVMYKGSPTVDSQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.814617534 (566h23m46.307049965s) [noise=true , address=77.90.31.15:4697 , piece_id=NBBDIDC64NAZKUYUGLHV5DAHDZKHHDHDWS3OBQG3VF5GOUHZVPWA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:41.942437754 (566h24m2.43487019s) [noise=true , address=145.79.227.18:4005 , piece_id=IMNWS75R4MTBLDUVYR6TCOWZLB5U3763R56HKKO3LSKGJVPSIGNQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:31:54.607167993 (566h24m15.099600436s) [noise=true , address=145.79.227.17:4970 , piece_id=A5S2OJM5VUCSEZAJFOWHQK2XUKXUTT5VVSQLKK6ZC62ARU6WZSPQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607533226 (566h24m15.099965694s) [noise=true , address=145.79.227.17:4971 , piece_id=RWVET42YTHRDPI7D6GRSYUK7EIDRSWCZ5VNDIKCA4VRMGUCVBAFQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607775799 (566h24m15.100208249s) [noise=true , address=145.79.227.2:4115 , piece_id=4BO6TSTGW7MMJTLXNQFU55C7I2OBGPJZWQKW4R5U2SPAWQS6IHPA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607801850 (566h24m15.100234304s) [seg_pos={0 0}, upload=498269]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:31:54.718884679 (566h24m15.211317115s) [upload=498269]: Done uploading segments... Sep 22 04:31:54.718934548 (566h24m15.211366981s) [upload=498269]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.400802656s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [X..XXXX.........................XX..X..X......X.............] duration: 2.642582888s FAIL (2025-09-22 04:34:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.642614s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [.X..XX..X......XXX.....XX.X..X.XX..X..X.X.............X.....] duration: 2.491311453s FAIL (2025-09-22 04:34:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:31:24.920143949 (566h23m45.41257638s) [upload=498269]: Starting upload Sep 22 04:31:24.920215948 (566h23m45.412648393s) [upload=498269]: Done starting upload Sep 22 04:31:24.920362595 (566h23m45.412795038s) [upload=498269]: Uploading segments... Sep 22 04:31:24.920492684 (566h23m45.412925131s) [upload=498269]: Got next segment. Inline: false Sep 22 04:31:24.920572454 (566h23m45.41300489s) [upload=498269]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:31:24.920578942 (566h23m45.413011385s) [upload=498269]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:31:25.045745592 (566h23m45.53817805s) [seg_pos={0 0}, upload=498269]: Begin upload segment... Sep 22 04:31:25.045861133 (566h23m45.538293585s) [seg_pos={0 0}, upload=498269]: Done begin upload segment. Sep 22 04:31:25.045945099 (566h23m45.53837755s) [noise=true , address=45.91.116.9:4261 , piece_id=CEGMAKDYVS3WEZVJLWZ7PLPIFF5HGU72XR3RYN5KFI6JYY6773MQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.045969575 (566h23m45.538402017s) [noise=true , address=145.79.227.2:4115 , piece_id=4BO6TSTGW7MMJTLXNQFU55C7I2OBGPJZWQKW4R5U2SPAWQS6IHPA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046521915 (566h23m45.538954365s) [noise=true , address=45.91.116.13:4155 , piece_id=7S37SPYHOEXLMC6QHLGZLH7KRXD7SCV3U4LQ62OBO2GOEBMY4DXA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046575700 (566h23m45.539008155s) [noise=true , address=145.79.227.18:4005 , piece_id=IMNWS75R4MTBLDUVYR6TCOWZLB5U3763R56HKKO3LSKGJVPSIGNQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046904954 (566h23m45.539337399s) [noise=true , address=45.91.116.8:4045 , piece_id=WQBNA3NDZBQ6NAMLMB4IKGOZIH4SHZACNOSMRQBUJXMSO4AKESVA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047049339 (566h23m45.53948179s) [noise=true , address=45.91.116.11:4117 , piece_id=2ZNQUGE6R2QOZJCRA4KWT2Y7IKZ5ZPQWTKNTSMK3WOZ4KUN5KQZQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047469004 (566h23m45.539901457s) [noise=true , address=77.90.31.19:4841 , piece_id=ZFNAUEMLT6VWPVIP53EKINHKNWC4H7UNJSTL2OYUS6WPFAUOPS5A, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047957742 (566h23m45.540390184s) [noise=true , address=77.90.31.12:4589 , piece_id=J2VRYHV43YRSOJ2X7NXRSO35O4JNBJELON27GFGJBSQG3L7DIKUA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048390500 (566h23m45.540822989s) [noise=true , address=145.79.227.17:4970 , piece_id=A5S2OJM5VUCSEZAJFOWHQK2XUKXUTT5VVSQLKK6ZC62ARU6WZSPQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048619410 (566h23m45.541051859s) [noise=true , address=77.90.31.15:4697 , piece_id=NBBDIDC64NAZKUYUGLHV5DAHDZKHHDHDWS3OBQG3VF5GOUHZVPWA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048962659 (566h23m45.541395101s) [noise=true , address=45.91.116.7:4010 , piece_id=OWKSKTCLX3JFSDOCEP37IHTAO32TICL7F7YMSK6KIH47ZRGJIT5Q, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049294858 (566h23m45.541727303s) [noise=true , address=77.90.31.16:4734 , piece_id=6PPHPDJRMRKNEYHMDDZVCLICO5KUXMZAFBZ7EF6XVMYKGSPTVDSQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049592400 (566h23m45.542024845s) [noise=true , address=77.90.31.10:4520 , piece_id=AGNOJKORSH4CEZUJ2P6CFFHVNWK6IB2S5GXUSXGU4ALMPZJCFTQQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049939474 (566h23m45.542371921s) [noise=true , address=145.79.227.17:4971 , piece_id=RWVET42YTHRDPI7D6GRSYUK7EIDRSWCZ5VNDIKCA4VRMGUCVBAFQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.050366023 (566h23m45.542798478s) [noise=true , address=45.91.116.14:4190 , piece_id=TS3M625BQNTBC4YBH3GOVRJ323CBRV34K3CAJMX7FNSQBYWSSOYQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.050850463 (566h23m45.543282914s) [noise=true , address=77.90.31.18:4805 , piece_id=F5IQFMGK7T5IGFALDPPMC4QBKMFUQ2OG7622QA2YJBUQDWBS4CVA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.667633076 (566h23m46.160065509s) [upload=498269]: Next returned nil segment Sep 22 04:31:25.667664989 (566h23m46.160097432s) [upload=498269]: Waiting for error group managing segments... Sep 22 04:31:25.803338183 (566h23m46.295770615s) [noise=true , address=45.91.116.8:4045 , piece_id=WQBNA3NDZBQ6NAMLMB4IKGOZIH4SHZACNOSMRQBUJXMSO4AKESVA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.805064032 (566h23m46.297496465s) [noise=true , address=77.90.31.10:4520 , piece_id=AGNOJKORSH4CEZUJ2P6CFFHVNWK6IB2S5GXUSXGU4ALMPZJCFTQQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.806116220 (566h23m46.298548651s) [noise=true , address=45.91.116.7:4010 , piece_id=OWKSKTCLX3JFSDOCEP37IHTAO32TICL7F7YMSK6KIH47ZRGJIT5Q, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.806176194 (566h23m46.298608727s) [noise=true , address=45.91.116.9:4261 , piece_id=CEGMAKDYVS3WEZVJLWZ7PLPIFF5HGU72XR3RYN5KFI6JYY6773MQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.809766126 (566h23m46.30219857s) [noise=true , address=45.91.116.14:4190 , piece_id=TS3M625BQNTBC4YBH3GOVRJ323CBRV34K3CAJMX7FNSQBYWSSOYQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.810019259 (566h23m46.302451696s) [noise=true , address=45.91.116.11:4117 , piece_id=2ZNQUGE6R2QOZJCRA4KWT2Y7IKZ5ZPQWTKNTSMK3WOZ4KUN5KQZQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.810642598 (566h23m46.303075034s) [noise=true , address=77.90.31.12:4589 , piece_id=J2VRYHV43YRSOJ2X7NXRSO35O4JNBJELON27GFGJBSQG3L7DIKUA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.811792355 (566h23m46.304224799s) [noise=true , address=77.90.31.19:4841 , piece_id=ZFNAUEMLT6VWPVIP53EKINHKNWC4H7UNJSTL2OYUS6WPFAUOPS5A, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.811940311 (566h23m46.304372744s) [noise=true , address=45.91.116.13:4155 , piece_id=7S37SPYHOEXLMC6QHLGZLH7KRXD7SCV3U4LQ62OBO2GOEBMY4DXA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.812546498 (566h23m46.304978934s) [noise=true , address=77.90.31.18:4805 , piece_id=F5IQFMGK7T5IGFALDPPMC4QBKMFUQ2OG7622QA2YJBUQDWBS4CVA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.813702226 (566h23m46.306134661s) [noise=true , address=77.90.31.16:4734 , piece_id=6PPHPDJRMRKNEYHMDDZVCLICO5KUXMZAFBZ7EF6XVMYKGSPTVDSQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.814617534 (566h23m46.307049965s) [noise=true , address=77.90.31.15:4697 , piece_id=NBBDIDC64NAZKUYUGLHV5DAHDZKHHDHDWS3OBQG3VF5GOUHZVPWA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:41.942437754 (566h24m2.43487019s) [noise=true , address=145.79.227.18:4005 , piece_id=IMNWS75R4MTBLDUVYR6TCOWZLB5U3763R56HKKO3LSKGJVPSIGNQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:31:54.607167993 (566h24m15.099600436s) [noise=true , address=145.79.227.17:4970 , piece_id=A5S2OJM5VUCSEZAJFOWHQK2XUKXUTT5VVSQLKK6ZC62ARU6WZSPQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607533226 (566h24m15.099965694s) [noise=true , address=145.79.227.17:4971 , piece_id=RWVET42YTHRDPI7D6GRSYUK7EIDRSWCZ5VNDIKCA4VRMGUCVBAFQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607775799 (566h24m15.100208249s) [noise=true , address=145.79.227.2:4115 , piece_id=4BO6TSTGW7MMJTLXNQFU55C7I2OBGPJZWQKW4R5U2SPAWQS6IHPA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607801850 (566h24m15.100234304s) [seg_pos={0 0}, upload=498269]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:31:54.718884679 (566h24m15.211317115s) [upload=498269]: Done uploading segments... Sep 22 04:31:54.718934548 (566h24m15.211366981s) [upload=498269]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.491350436s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.X..XXXX.........................XX..X..X......X............] duration: 2.708493807s FAIL (2025-09-22 04:33:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.708537289s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [..X..XX..X......XXX.....XX.X..X.XX..X..X.X.............X....] duration: 2.744246355s FAIL (2025-09-22 04:33:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:31:24.920143949 (566h23m45.41257638s) [upload=498269]: Starting upload Sep 22 04:31:24.920215948 (566h23m45.412648393s) [upload=498269]: Done starting upload Sep 22 04:31:24.920362595 (566h23m45.412795038s) [upload=498269]: Uploading segments... Sep 22 04:31:24.920492684 (566h23m45.412925131s) [upload=498269]: Got next segment. Inline: false Sep 22 04:31:24.920572454 (566h23m45.41300489s) [upload=498269]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:31:24.920578942 (566h23m45.413011385s) [upload=498269]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:31:25.045745592 (566h23m45.53817805s) [seg_pos={0 0}, upload=498269]: Begin upload segment... Sep 22 04:31:25.045861133 (566h23m45.538293585s) [seg_pos={0 0}, upload=498269]: Done begin upload segment. Sep 22 04:31:25.045945099 (566h23m45.53837755s) [noise=true , address=45.91.116.9:4261 , piece_id=CEGMAKDYVS3WEZVJLWZ7PLPIFF5HGU72XR3RYN5KFI6JYY6773MQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.045969575 (566h23m45.538402017s) [noise=true , address=145.79.227.2:4115 , piece_id=4BO6TSTGW7MMJTLXNQFU55C7I2OBGPJZWQKW4R5U2SPAWQS6IHPA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046521915 (566h23m45.538954365s) [noise=true , address=45.91.116.13:4155 , piece_id=7S37SPYHOEXLMC6QHLGZLH7KRXD7SCV3U4LQ62OBO2GOEBMY4DXA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046575700 (566h23m45.539008155s) [noise=true , address=145.79.227.18:4005 , piece_id=IMNWS75R4MTBLDUVYR6TCOWZLB5U3763R56HKKO3LSKGJVPSIGNQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046904954 (566h23m45.539337399s) [noise=true , address=45.91.116.8:4045 , piece_id=WQBNA3NDZBQ6NAMLMB4IKGOZIH4SHZACNOSMRQBUJXMSO4AKESVA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047049339 (566h23m45.53948179s) [noise=true , address=45.91.116.11:4117 , piece_id=2ZNQUGE6R2QOZJCRA4KWT2Y7IKZ5ZPQWTKNTSMK3WOZ4KUN5KQZQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047469004 (566h23m45.539901457s) [noise=true , address=77.90.31.19:4841 , piece_id=ZFNAUEMLT6VWPVIP53EKINHKNWC4H7UNJSTL2OYUS6WPFAUOPS5A, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047957742 (566h23m45.540390184s) [noise=true , address=77.90.31.12:4589 , piece_id=J2VRYHV43YRSOJ2X7NXRSO35O4JNBJELON27GFGJBSQG3L7DIKUA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048390500 (566h23m45.540822989s) [noise=true , address=145.79.227.17:4970 , piece_id=A5S2OJM5VUCSEZAJFOWHQK2XUKXUTT5VVSQLKK6ZC62ARU6WZSPQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048619410 (566h23m45.541051859s) [noise=true , address=77.90.31.15:4697 , piece_id=NBBDIDC64NAZKUYUGLHV5DAHDZKHHDHDWS3OBQG3VF5GOUHZVPWA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048962659 (566h23m45.541395101s) [noise=true , address=45.91.116.7:4010 , piece_id=OWKSKTCLX3JFSDOCEP37IHTAO32TICL7F7YMSK6KIH47ZRGJIT5Q, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049294858 (566h23m45.541727303s) [noise=true , address=77.90.31.16:4734 , piece_id=6PPHPDJRMRKNEYHMDDZVCLICO5KUXMZAFBZ7EF6XVMYKGSPTVDSQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049592400 (566h23m45.542024845s) [noise=true , address=77.90.31.10:4520 , piece_id=AGNOJKORSH4CEZUJ2P6CFFHVNWK6IB2S5GXUSXGU4ALMPZJCFTQQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049939474 (566h23m45.542371921s) [noise=true , address=145.79.227.17:4971 , piece_id=RWVET42YTHRDPI7D6GRSYUK7EIDRSWCZ5VNDIKCA4VRMGUCVBAFQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.050366023 (566h23m45.542798478s) [noise=true , address=45.91.116.14:4190 , piece_id=TS3M625BQNTBC4YBH3GOVRJ323CBRV34K3CAJMX7FNSQBYWSSOYQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.050850463 (566h23m45.543282914s) [noise=true , address=77.90.31.18:4805 , piece_id=F5IQFMGK7T5IGFALDPPMC4QBKMFUQ2OG7622QA2YJBUQDWBS4CVA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.667633076 (566h23m46.160065509s) [upload=498269]: Next returned nil segment Sep 22 04:31:25.667664989 (566h23m46.160097432s) [upload=498269]: Waiting for error group managing segments... Sep 22 04:31:25.803338183 (566h23m46.295770615s) [noise=true , address=45.91.116.8:4045 , piece_id=WQBNA3NDZBQ6NAMLMB4IKGOZIH4SHZACNOSMRQBUJXMSO4AKESVA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.805064032 (566h23m46.297496465s) [noise=true , address=77.90.31.10:4520 , piece_id=AGNOJKORSH4CEZUJ2P6CFFHVNWK6IB2S5GXUSXGU4ALMPZJCFTQQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.806116220 (566h23m46.298548651s) [noise=true , address=45.91.116.7:4010 , piece_id=OWKSKTCLX3JFSDOCEP37IHTAO32TICL7F7YMSK6KIH47ZRGJIT5Q, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.806176194 (566h23m46.298608727s) [noise=true , address=45.91.116.9:4261 , piece_id=CEGMAKDYVS3WEZVJLWZ7PLPIFF5HGU72XR3RYN5KFI6JYY6773MQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.809766126 (566h23m46.30219857s) [noise=true , address=45.91.116.14:4190 , piece_id=TS3M625BQNTBC4YBH3GOVRJ323CBRV34K3CAJMX7FNSQBYWSSOYQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.810019259 (566h23m46.302451696s) [noise=true , address=45.91.116.11:4117 , piece_id=2ZNQUGE6R2QOZJCRA4KWT2Y7IKZ5ZPQWTKNTSMK3WOZ4KUN5KQZQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.810642598 (566h23m46.303075034s) [noise=true , address=77.90.31.12:4589 , piece_id=J2VRYHV43YRSOJ2X7NXRSO35O4JNBJELON27GFGJBSQG3L7DIKUA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.811792355 (566h23m46.304224799s) [noise=true , address=77.90.31.19:4841 , piece_id=ZFNAUEMLT6VWPVIP53EKINHKNWC4H7UNJSTL2OYUS6WPFAUOPS5A, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.811940311 (566h23m46.304372744s) [noise=true , address=45.91.116.13:4155 , piece_id=7S37SPYHOEXLMC6QHLGZLH7KRXD7SCV3U4LQ62OBO2GOEBMY4DXA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.812546498 (566h23m46.304978934s) [noise=true , address=77.90.31.18:4805 , piece_id=F5IQFMGK7T5IGFALDPPMC4QBKMFUQ2OG7622QA2YJBUQDWBS4CVA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.813702226 (566h23m46.306134661s) [noise=true , address=77.90.31.16:4734 , piece_id=6PPHPDJRMRKNEYHMDDZVCLICO5KUXMZAFBZ7EF6XVMYKGSPTVDSQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.814617534 (566h23m46.307049965s) [noise=true , address=77.90.31.15:4697 , piece_id=NBBDIDC64NAZKUYUGLHV5DAHDZKHHDHDWS3OBQG3VF5GOUHZVPWA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:41.942437754 (566h24m2.43487019s) [noise=true , address=145.79.227.18:4005 , piece_id=IMNWS75R4MTBLDUVYR6TCOWZLB5U3763R56HKKO3LSKGJVPSIGNQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:31:54.607167993 (566h24m15.099600436s) [noise=true , address=145.79.227.17:4970 , piece_id=A5S2OJM5VUCSEZAJFOWHQK2XUKXUTT5VVSQLKK6ZC62ARU6WZSPQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607533226 (566h24m15.099965694s) [noise=true , address=145.79.227.17:4971 , piece_id=RWVET42YTHRDPI7D6GRSYUK7EIDRSWCZ5VNDIKCA4VRMGUCVBAFQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607775799 (566h24m15.100208249s) [noise=true , address=145.79.227.2:4115 , piece_id=4BO6TSTGW7MMJTLXNQFU55C7I2OBGPJZWQKW4R5U2SPAWQS6IHPA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607801850 (566h24m15.100234304s) [seg_pos={0 0}, upload=498269]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:31:54.718884679 (566h24m15.211317115s) [upload=498269]: Done uploading segments... Sep 22 04:31:54.718934548 (566h24m15.211366981s) [upload=498269]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.744280484s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [X..X..XX..X......XXX.....XX.X..X.XX..X..X.X.............X...] duration: 2.417876991s FAIL (2025-09-22 04:33:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:31:24.920143949 (566h23m45.41257638s) [upload=498269]: Starting upload Sep 22 04:31:24.920215948 (566h23m45.412648393s) [upload=498269]: Done starting upload Sep 22 04:31:24.920362595 (566h23m45.412795038s) [upload=498269]: Uploading segments... Sep 22 04:31:24.920492684 (566h23m45.412925131s) [upload=498269]: Got next segment. Inline: false Sep 22 04:31:24.920572454 (566h23m45.41300489s) [upload=498269]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:31:24.920578942 (566h23m45.413011385s) [upload=498269]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:31:25.045745592 (566h23m45.53817805s) [seg_pos={0 0}, upload=498269]: Begin upload segment... Sep 22 04:31:25.045861133 (566h23m45.538293585s) [seg_pos={0 0}, upload=498269]: Done begin upload segment. Sep 22 04:31:25.045945099 (566h23m45.53837755s) [noise=true , address=45.91.116.9:4261 , piece_id=CEGMAKDYVS3WEZVJLWZ7PLPIFF5HGU72XR3RYN5KFI6JYY6773MQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.045969575 (566h23m45.538402017s) [noise=true , address=145.79.227.2:4115 , piece_id=4BO6TSTGW7MMJTLXNQFU55C7I2OBGPJZWQKW4R5U2SPAWQS6IHPA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046521915 (566h23m45.538954365s) [noise=true , address=45.91.116.13:4155 , piece_id=7S37SPYHOEXLMC6QHLGZLH7KRXD7SCV3U4LQ62OBO2GOEBMY4DXA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046575700 (566h23m45.539008155s) [noise=true , address=145.79.227.18:4005 , piece_id=IMNWS75R4MTBLDUVYR6TCOWZLB5U3763R56HKKO3LSKGJVPSIGNQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046904954 (566h23m45.539337399s) [noise=true , address=45.91.116.8:4045 , piece_id=WQBNA3NDZBQ6NAMLMB4IKGOZIH4SHZACNOSMRQBUJXMSO4AKESVA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047049339 (566h23m45.53948179s) [noise=true , address=45.91.116.11:4117 , piece_id=2ZNQUGE6R2QOZJCRA4KWT2Y7IKZ5ZPQWTKNTSMK3WOZ4KUN5KQZQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047469004 (566h23m45.539901457s) [noise=true , address=77.90.31.19:4841 , piece_id=ZFNAUEMLT6VWPVIP53EKINHKNWC4H7UNJSTL2OYUS6WPFAUOPS5A, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047957742 (566h23m45.540390184s) [noise=true , address=77.90.31.12:4589 , piece_id=J2VRYHV43YRSOJ2X7NXRSO35O4JNBJELON27GFGJBSQG3L7DIKUA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048390500 (566h23m45.540822989s) [noise=true , address=145.79.227.17:4970 , piece_id=A5S2OJM5VUCSEZAJFOWHQK2XUKXUTT5VVSQLKK6ZC62ARU6WZSPQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048619410 (566h23m45.541051859s) [noise=true , address=77.90.31.15:4697 , piece_id=NBBDIDC64NAZKUYUGLHV5DAHDZKHHDHDWS3OBQG3VF5GOUHZVPWA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048962659 (566h23m45.541395101s) [noise=true , address=45.91.116.7:4010 , piece_id=OWKSKTCLX3JFSDOCEP37IHTAO32TICL7F7YMSK6KIH47ZRGJIT5Q, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049294858 (566h23m45.541727303s) [noise=true , address=77.90.31.16:4734 , piece_id=6PPHPDJRMRKNEYHMDDZVCLICO5KUXMZAFBZ7EF6XVMYKGSPTVDSQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049592400 (566h23m45.542024845s) [noise=true , address=77.90.31.10:4520 , piece_id=AGNOJKORSH4CEZUJ2P6CFFHVNWK6IB2S5GXUSXGU4ALMPZJCFTQQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049939474 (566h23m45.542371921s) [noise=true , address=145.79.227.17:4971 , piece_id=RWVET42YTHRDPI7D6GRSYUK7EIDRSWCZ5VNDIKCA4VRMGUCVBAFQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.050366023 (566h23m45.542798478s) [noise=true , address=45.91.116.14:4190 , piece_id=TS3M625BQNTBC4YBH3GOVRJ323CBRV34K3CAJMX7FNSQBYWSSOYQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.050850463 (566h23m45.543282914s) [noise=true , address=77.90.31.18:4805 , piece_id=F5IQFMGK7T5IGFALDPPMC4QBKMFUQ2OG7622QA2YJBUQDWBS4CVA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.667633076 (566h23m46.160065509s) [upload=498269]: Next returned nil segment Sep 22 04:31:25.667664989 (566h23m46.160097432s) [upload=498269]: Waiting for error group managing segments... Sep 22 04:31:25.803338183 (566h23m46.295770615s) [noise=true , address=45.91.116.8:4045 , piece_id=WQBNA3NDZBQ6NAMLMB4IKGOZIH4SHZACNOSMRQBUJXMSO4AKESVA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.805064032 (566h23m46.297496465s) [noise=true , address=77.90.31.10:4520 , piece_id=AGNOJKORSH4CEZUJ2P6CFFHVNWK6IB2S5GXUSXGU4ALMPZJCFTQQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.806116220 (566h23m46.298548651s) [noise=true , address=45.91.116.7:4010 , piece_id=OWKSKTCLX3JFSDOCEP37IHTAO32TICL7F7YMSK6KIH47ZRGJIT5Q, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.806176194 (566h23m46.298608727s) [noise=true , address=45.91.116.9:4261 , piece_id=CEGMAKDYVS3WEZVJLWZ7PLPIFF5HGU72XR3RYN5KFI6JYY6773MQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.809766126 (566h23m46.30219857s) [noise=true , address=45.91.116.14:4190 , piece_id=TS3M625BQNTBC4YBH3GOVRJ323CBRV34K3CAJMX7FNSQBYWSSOYQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.810019259 (566h23m46.302451696s) [noise=true , address=45.91.116.11:4117 , piece_id=2ZNQUGE6R2QOZJCRA4KWT2Y7IKZ5ZPQWTKNTSMK3WOZ4KUN5KQZQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.810642598 (566h23m46.303075034s) [noise=true , address=77.90.31.12:4589 , piece_id=J2VRYHV43YRSOJ2X7NXRSO35O4JNBJELON27GFGJBSQG3L7DIKUA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.811792355 (566h23m46.304224799s) [noise=true , address=77.90.31.19:4841 , piece_id=ZFNAUEMLT6VWPVIP53EKINHKNWC4H7UNJSTL2OYUS6WPFAUOPS5A, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.811940311 (566h23m46.304372744s) [noise=true , address=45.91.116.13:4155 , piece_id=7S37SPYHOEXLMC6QHLGZLH7KRXD7SCV3U4LQ62OBO2GOEBMY4DXA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.812546498 (566h23m46.304978934s) [noise=true , address=77.90.31.18:4805 , piece_id=F5IQFMGK7T5IGFALDPPMC4QBKMFUQ2OG7622QA2YJBUQDWBS4CVA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.813702226 (566h23m46.306134661s) [noise=true , address=77.90.31.16:4734 , piece_id=6PPHPDJRMRKNEYHMDDZVCLICO5KUXMZAFBZ7EF6XVMYKGSPTVDSQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.814617534 (566h23m46.307049965s) [noise=true , address=77.90.31.15:4697 , piece_id=NBBDIDC64NAZKUYUGLHV5DAHDZKHHDHDWS3OBQG3VF5GOUHZVPWA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:41.942437754 (566h24m2.43487019s) [noise=true , address=145.79.227.18:4005 , piece_id=IMNWS75R4MTBLDUVYR6TCOWZLB5U3763R56HKKO3LSKGJVPSIGNQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:31:54.607167993 (566h24m15.099600436s) [noise=true , address=145.79.227.17:4970 , piece_id=A5S2OJM5VUCSEZAJFOWHQK2XUKXUTT5VVSQLKK6ZC62ARU6WZSPQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607533226 (566h24m15.099965694s) [noise=true , address=145.79.227.17:4971 , piece_id=RWVET42YTHRDPI7D6GRSYUK7EIDRSWCZ5VNDIKCA4VRMGUCVBAFQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607775799 (566h24m15.100208249s) [noise=true , address=145.79.227.2:4115 , piece_id=4BO6TSTGW7MMJTLXNQFU55C7I2OBGPJZWQKW4R5U2SPAWQS6IHPA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607801850 (566h24m15.100234304s) [seg_pos={0 0}, upload=498269]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:31:54.718884679 (566h24m15.211317115s) [upload=498269]: Done uploading segments... Sep 22 04:31:54.718934548 (566h24m15.211366981s) [upload=498269]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.417900948s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..X..XXXX.........................XX..X..X......X...........] duration: 2.665822397s FAIL (2025-09-22 04:32:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.66584614s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [.X..X..XX..X......XXX.....XX.X..X.XX..X..X.X.............X..] duration: 2.377074752s FAIL (2025-09-22 04:32:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:31:24.920143949 (566h23m45.41257638s) [upload=498269]: Starting upload Sep 22 04:31:24.920215948 (566h23m45.412648393s) [upload=498269]: Done starting upload Sep 22 04:31:24.920362595 (566h23m45.412795038s) [upload=498269]: Uploading segments... Sep 22 04:31:24.920492684 (566h23m45.412925131s) [upload=498269]: Got next segment. Inline: false Sep 22 04:31:24.920572454 (566h23m45.41300489s) [upload=498269]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:31:24.920578942 (566h23m45.413011385s) [upload=498269]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:31:25.045745592 (566h23m45.53817805s) [seg_pos={0 0}, upload=498269]: Begin upload segment... Sep 22 04:31:25.045861133 (566h23m45.538293585s) [seg_pos={0 0}, upload=498269]: Done begin upload segment. Sep 22 04:31:25.045945099 (566h23m45.53837755s) [noise=true , address=45.91.116.9:4261 , piece_id=CEGMAKDYVS3WEZVJLWZ7PLPIFF5HGU72XR3RYN5KFI6JYY6773MQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.045969575 (566h23m45.538402017s) [noise=true , address=145.79.227.2:4115 , piece_id=4BO6TSTGW7MMJTLXNQFU55C7I2OBGPJZWQKW4R5U2SPAWQS6IHPA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046521915 (566h23m45.538954365s) [noise=true , address=45.91.116.13:4155 , piece_id=7S37SPYHOEXLMC6QHLGZLH7KRXD7SCV3U4LQ62OBO2GOEBMY4DXA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046575700 (566h23m45.539008155s) [noise=true , address=145.79.227.18:4005 , piece_id=IMNWS75R4MTBLDUVYR6TCOWZLB5U3763R56HKKO3LSKGJVPSIGNQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046904954 (566h23m45.539337399s) [noise=true , address=45.91.116.8:4045 , piece_id=WQBNA3NDZBQ6NAMLMB4IKGOZIH4SHZACNOSMRQBUJXMSO4AKESVA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047049339 (566h23m45.53948179s) [noise=true , address=45.91.116.11:4117 , piece_id=2ZNQUGE6R2QOZJCRA4KWT2Y7IKZ5ZPQWTKNTSMK3WOZ4KUN5KQZQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047469004 (566h23m45.539901457s) [noise=true , address=77.90.31.19:4841 , piece_id=ZFNAUEMLT6VWPVIP53EKINHKNWC4H7UNJSTL2OYUS6WPFAUOPS5A, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047957742 (566h23m45.540390184s) [noise=true , address=77.90.31.12:4589 , piece_id=J2VRYHV43YRSOJ2X7NXRSO35O4JNBJELON27GFGJBSQG3L7DIKUA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048390500 (566h23m45.540822989s) [noise=true , address=145.79.227.17:4970 , piece_id=A5S2OJM5VUCSEZAJFOWHQK2XUKXUTT5VVSQLKK6ZC62ARU6WZSPQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048619410 (566h23m45.541051859s) [noise=true , address=77.90.31.15:4697 , piece_id=NBBDIDC64NAZKUYUGLHV5DAHDZKHHDHDWS3OBQG3VF5GOUHZVPWA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048962659 (566h23m45.541395101s) [noise=true , address=45.91.116.7:4010 , piece_id=OWKSKTCLX3JFSDOCEP37IHTAO32TICL7F7YMSK6KIH47ZRGJIT5Q, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049294858 (566h23m45.541727303s) [noise=true , address=77.90.31.16:4734 , piece_id=6PPHPDJRMRKNEYHMDDZVCLICO5KUXMZAFBZ7EF6XVMYKGSPTVDSQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049592400 (566h23m45.542024845s) [noise=true , address=77.90.31.10:4520 , piece_id=AGNOJKORSH4CEZUJ2P6CFFHVNWK6IB2S5GXUSXGU4ALMPZJCFTQQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049939474 (566h23m45.542371921s) [noise=true , address=145.79.227.17:4971 , piece_id=RWVET42YTHRDPI7D6GRSYUK7EIDRSWCZ5VNDIKCA4VRMGUCVBAFQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.050366023 (566h23m45.542798478s) [noise=true , address=45.91.116.14:4190 , piece_id=TS3M625BQNTBC4YBH3GOVRJ323CBRV34K3CAJMX7FNSQBYWSSOYQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.050850463 (566h23m45.543282914s) [noise=true , address=77.90.31.18:4805 , piece_id=F5IQFMGK7T5IGFALDPPMC4QBKMFUQ2OG7622QA2YJBUQDWBS4CVA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.667633076 (566h23m46.160065509s) [upload=498269]: Next returned nil segment Sep 22 04:31:25.667664989 (566h23m46.160097432s) [upload=498269]: Waiting for error group managing segments... Sep 22 04:31:25.803338183 (566h23m46.295770615s) [noise=true , address=45.91.116.8:4045 , piece_id=WQBNA3NDZBQ6NAMLMB4IKGOZIH4SHZACNOSMRQBUJXMSO4AKESVA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.805064032 (566h23m46.297496465s) [noise=true , address=77.90.31.10:4520 , piece_id=AGNOJKORSH4CEZUJ2P6CFFHVNWK6IB2S5GXUSXGU4ALMPZJCFTQQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.806116220 (566h23m46.298548651s) [noise=true , address=45.91.116.7:4010 , piece_id=OWKSKTCLX3JFSDOCEP37IHTAO32TICL7F7YMSK6KIH47ZRGJIT5Q, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.806176194 (566h23m46.298608727s) [noise=true , address=45.91.116.9:4261 , piece_id=CEGMAKDYVS3WEZVJLWZ7PLPIFF5HGU72XR3RYN5KFI6JYY6773MQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.809766126 (566h23m46.30219857s) [noise=true , address=45.91.116.14:4190 , piece_id=TS3M625BQNTBC4YBH3GOVRJ323CBRV34K3CAJMX7FNSQBYWSSOYQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.810019259 (566h23m46.302451696s) [noise=true , address=45.91.116.11:4117 , piece_id=2ZNQUGE6R2QOZJCRA4KWT2Y7IKZ5ZPQWTKNTSMK3WOZ4KUN5KQZQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.810642598 (566h23m46.303075034s) [noise=true , address=77.90.31.12:4589 , piece_id=J2VRYHV43YRSOJ2X7NXRSO35O4JNBJELON27GFGJBSQG3L7DIKUA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.811792355 (566h23m46.304224799s) [noise=true , address=77.90.31.19:4841 , piece_id=ZFNAUEMLT6VWPVIP53EKINHKNWC4H7UNJSTL2OYUS6WPFAUOPS5A, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.811940311 (566h23m46.304372744s) [noise=true , address=45.91.116.13:4155 , piece_id=7S37SPYHOEXLMC6QHLGZLH7KRXD7SCV3U4LQ62OBO2GOEBMY4DXA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.812546498 (566h23m46.304978934s) [noise=true , address=77.90.31.18:4805 , piece_id=F5IQFMGK7T5IGFALDPPMC4QBKMFUQ2OG7622QA2YJBUQDWBS4CVA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.813702226 (566h23m46.306134661s) [noise=true , address=77.90.31.16:4734 , piece_id=6PPHPDJRMRKNEYHMDDZVCLICO5KUXMZAFBZ7EF6XVMYKGSPTVDSQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.814617534 (566h23m46.307049965s) [noise=true , address=77.90.31.15:4697 , piece_id=NBBDIDC64NAZKUYUGLHV5DAHDZKHHDHDWS3OBQG3VF5GOUHZVPWA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:41.942437754 (566h24m2.43487019s) [noise=true , address=145.79.227.18:4005 , piece_id=IMNWS75R4MTBLDUVYR6TCOWZLB5U3763R56HKKO3LSKGJVPSIGNQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:31:54.607167993 (566h24m15.099600436s) [noise=true , address=145.79.227.17:4970 , piece_id=A5S2OJM5VUCSEZAJFOWHQK2XUKXUTT5VVSQLKK6ZC62ARU6WZSPQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607533226 (566h24m15.099965694s) [noise=true , address=145.79.227.17:4971 , piece_id=RWVET42YTHRDPI7D6GRSYUK7EIDRSWCZ5VNDIKCA4VRMGUCVBAFQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607775799 (566h24m15.100208249s) [noise=true , address=145.79.227.2:4115 , piece_id=4BO6TSTGW7MMJTLXNQFU55C7I2OBGPJZWQKW4R5U2SPAWQS6IHPA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607801850 (566h24m15.100234304s) [seg_pos={0 0}, upload=498269]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:31:54.718884679 (566h24m15.211317115s) [upload=498269]: Done uploading segments... Sep 22 04:31:54.718934548 (566h24m15.211366981s) [upload=498269]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.377109522s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [..X..X..XX..X......XXX.....XX.X..X.XX..X..X.X.............X.] duration: 2.495877505s FAIL (2025-09-22 04:31:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:31:24.920143949 (566h23m45.41257638s) [upload=498269]: Starting upload Sep 22 04:31:24.920215948 (566h23m45.412648393s) [upload=498269]: Done starting upload Sep 22 04:31:24.920362595 (566h23m45.412795038s) [upload=498269]: Uploading segments... Sep 22 04:31:24.920492684 (566h23m45.412925131s) [upload=498269]: Got next segment. Inline: false Sep 22 04:31:24.920572454 (566h23m45.41300489s) [upload=498269]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:31:24.920578942 (566h23m45.413011385s) [upload=498269]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:31:25.045745592 (566h23m45.53817805s) [seg_pos={0 0}, upload=498269]: Begin upload segment... Sep 22 04:31:25.045861133 (566h23m45.538293585s) [seg_pos={0 0}, upload=498269]: Done begin upload segment. Sep 22 04:31:25.045945099 (566h23m45.53837755s) [noise=true , address=45.91.116.9:4261 , piece_id=CEGMAKDYVS3WEZVJLWZ7PLPIFF5HGU72XR3RYN5KFI6JYY6773MQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.045969575 (566h23m45.538402017s) [noise=true , address=145.79.227.2:4115 , piece_id=4BO6TSTGW7MMJTLXNQFU55C7I2OBGPJZWQKW4R5U2SPAWQS6IHPA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046521915 (566h23m45.538954365s) [noise=true , address=45.91.116.13:4155 , piece_id=7S37SPYHOEXLMC6QHLGZLH7KRXD7SCV3U4LQ62OBO2GOEBMY4DXA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046575700 (566h23m45.539008155s) [noise=true , address=145.79.227.18:4005 , piece_id=IMNWS75R4MTBLDUVYR6TCOWZLB5U3763R56HKKO3LSKGJVPSIGNQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046904954 (566h23m45.539337399s) [noise=true , address=45.91.116.8:4045 , piece_id=WQBNA3NDZBQ6NAMLMB4IKGOZIH4SHZACNOSMRQBUJXMSO4AKESVA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047049339 (566h23m45.53948179s) [noise=true , address=45.91.116.11:4117 , piece_id=2ZNQUGE6R2QOZJCRA4KWT2Y7IKZ5ZPQWTKNTSMK3WOZ4KUN5KQZQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047469004 (566h23m45.539901457s) [noise=true , address=77.90.31.19:4841 , piece_id=ZFNAUEMLT6VWPVIP53EKINHKNWC4H7UNJSTL2OYUS6WPFAUOPS5A, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047957742 (566h23m45.540390184s) [noise=true , address=77.90.31.12:4589 , piece_id=J2VRYHV43YRSOJ2X7NXRSO35O4JNBJELON27GFGJBSQG3L7DIKUA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048390500 (566h23m45.540822989s) [noise=true , address=145.79.227.17:4970 , piece_id=A5S2OJM5VUCSEZAJFOWHQK2XUKXUTT5VVSQLKK6ZC62ARU6WZSPQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048619410 (566h23m45.541051859s) [noise=true , address=77.90.31.15:4697 , piece_id=NBBDIDC64NAZKUYUGLHV5DAHDZKHHDHDWS3OBQG3VF5GOUHZVPWA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048962659 (566h23m45.541395101s) [noise=true , address=45.91.116.7:4010 , piece_id=OWKSKTCLX3JFSDOCEP37IHTAO32TICL7F7YMSK6KIH47ZRGJIT5Q, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049294858 (566h23m45.541727303s) [noise=true , address=77.90.31.16:4734 , piece_id=6PPHPDJRMRKNEYHMDDZVCLICO5KUXMZAFBZ7EF6XVMYKGSPTVDSQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049592400 (566h23m45.542024845s) [noise=true , address=77.90.31.10:4520 , piece_id=AGNOJKORSH4CEZUJ2P6CFFHVNWK6IB2S5GXUSXGU4ALMPZJCFTQQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049939474 (566h23m45.542371921s) [noise=true , address=145.79.227.17:4971 , piece_id=RWVET42YTHRDPI7D6GRSYUK7EIDRSWCZ5VNDIKCA4VRMGUCVBAFQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.050366023 (566h23m45.542798478s) [noise=true , address=45.91.116.14:4190 , piece_id=TS3M625BQNTBC4YBH3GOVRJ323CBRV34K3CAJMX7FNSQBYWSSOYQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.050850463 (566h23m45.543282914s) [noise=true , address=77.90.31.18:4805 , piece_id=F5IQFMGK7T5IGFALDPPMC4QBKMFUQ2OG7622QA2YJBUQDWBS4CVA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.667633076 (566h23m46.160065509s) [upload=498269]: Next returned nil segment Sep 22 04:31:25.667664989 (566h23m46.160097432s) [upload=498269]: Waiting for error group managing segments... Sep 22 04:31:25.803338183 (566h23m46.295770615s) [noise=true , address=45.91.116.8:4045 , piece_id=WQBNA3NDZBQ6NAMLMB4IKGOZIH4SHZACNOSMRQBUJXMSO4AKESVA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.805064032 (566h23m46.297496465s) [noise=true , address=77.90.31.10:4520 , piece_id=AGNOJKORSH4CEZUJ2P6CFFHVNWK6IB2S5GXUSXGU4ALMPZJCFTQQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.806116220 (566h23m46.298548651s) [noise=true , address=45.91.116.7:4010 , piece_id=OWKSKTCLX3JFSDOCEP37IHTAO32TICL7F7YMSK6KIH47ZRGJIT5Q, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.806176194 (566h23m46.298608727s) [noise=true , address=45.91.116.9:4261 , piece_id=CEGMAKDYVS3WEZVJLWZ7PLPIFF5HGU72XR3RYN5KFI6JYY6773MQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.809766126 (566h23m46.30219857s) [noise=true , address=45.91.116.14:4190 , piece_id=TS3M625BQNTBC4YBH3GOVRJ323CBRV34K3CAJMX7FNSQBYWSSOYQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.810019259 (566h23m46.302451696s) [noise=true , address=45.91.116.11:4117 , piece_id=2ZNQUGE6R2QOZJCRA4KWT2Y7IKZ5ZPQWTKNTSMK3WOZ4KUN5KQZQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.810642598 (566h23m46.303075034s) [noise=true , address=77.90.31.12:4589 , piece_id=J2VRYHV43YRSOJ2X7NXRSO35O4JNBJELON27GFGJBSQG3L7DIKUA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.811792355 (566h23m46.304224799s) [noise=true , address=77.90.31.19:4841 , piece_id=ZFNAUEMLT6VWPVIP53EKINHKNWC4H7UNJSTL2OYUS6WPFAUOPS5A, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.811940311 (566h23m46.304372744s) [noise=true , address=45.91.116.13:4155 , piece_id=7S37SPYHOEXLMC6QHLGZLH7KRXD7SCV3U4LQ62OBO2GOEBMY4DXA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.812546498 (566h23m46.304978934s) [noise=true , address=77.90.31.18:4805 , piece_id=F5IQFMGK7T5IGFALDPPMC4QBKMFUQ2OG7622QA2YJBUQDWBS4CVA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.813702226 (566h23m46.306134661s) [noise=true , address=77.90.31.16:4734 , piece_id=6PPHPDJRMRKNEYHMDDZVCLICO5KUXMZAFBZ7EF6XVMYKGSPTVDSQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.814617534 (566h23m46.307049965s) [noise=true , address=77.90.31.15:4697 , piece_id=NBBDIDC64NAZKUYUGLHV5DAHDZKHHDHDWS3OBQG3VF5GOUHZVPWA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:41.942437754 (566h24m2.43487019s) [noise=true , address=145.79.227.18:4005 , piece_id=IMNWS75R4MTBLDUVYR6TCOWZLB5U3763R56HKKO3LSKGJVPSIGNQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:31:54.607167993 (566h24m15.099600436s) [noise=true , address=145.79.227.17:4970 , piece_id=A5S2OJM5VUCSEZAJFOWHQK2XUKXUTT5VVSQLKK6ZC62ARU6WZSPQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607533226 (566h24m15.099965694s) [noise=true , address=145.79.227.17:4971 , piece_id=RWVET42YTHRDPI7D6GRSYUK7EIDRSWCZ5VNDIKCA4VRMGUCVBAFQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607775799 (566h24m15.100208249s) [noise=true , address=145.79.227.2:4115 , piece_id=4BO6TSTGW7MMJTLXNQFU55C7I2OBGPJZWQKW4R5U2SPAWQS6IHPA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607801850 (566h24m15.100234304s) [seg_pos={0 0}, upload=498269]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:31:54.718884679 (566h24m15.211317115s) [upload=498269]: Done uploading segments... Sep 22 04:31:54.718934548 (566h24m15.211366981s) [upload=498269]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.495913099s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...X..XXXX.........................XX..X..X......X..........] duration: 2.764999736s FAIL (2025-09-22 04:31:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.765053218s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [...X..X..XX..X......XXX.....XX.X..X.XX..X..X.X.............X] duration: 30.113948221s FAIL (2025-09-22 04:31:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:31:24.920143949 (566h23m45.41257638s) [upload=498269]: Starting upload Sep 22 04:31:24.920215948 (566h23m45.412648393s) [upload=498269]: Done starting upload Sep 22 04:31:24.920362595 (566h23m45.412795038s) [upload=498269]: Uploading segments... Sep 22 04:31:24.920492684 (566h23m45.412925131s) [upload=498269]: Got next segment. Inline: false Sep 22 04:31:24.920572454 (566h23m45.41300489s) [upload=498269]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:31:24.920578942 (566h23m45.413011385s) [upload=498269]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:31:25.045745592 (566h23m45.53817805s) [seg_pos={0 0}, upload=498269]: Begin upload segment... Sep 22 04:31:25.045861133 (566h23m45.538293585s) [seg_pos={0 0}, upload=498269]: Done begin upload segment. Sep 22 04:31:25.045945099 (566h23m45.53837755s) [noise=true , address=45.91.116.9:4261 , piece_id=CEGMAKDYVS3WEZVJLWZ7PLPIFF5HGU72XR3RYN5KFI6JYY6773MQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.045969575 (566h23m45.538402017s) [noise=true , address=145.79.227.2:4115 , piece_id=4BO6TSTGW7MMJTLXNQFU55C7I2OBGPJZWQKW4R5U2SPAWQS6IHPA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046521915 (566h23m45.538954365s) [noise=true , address=45.91.116.13:4155 , piece_id=7S37SPYHOEXLMC6QHLGZLH7KRXD7SCV3U4LQ62OBO2GOEBMY4DXA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046575700 (566h23m45.539008155s) [noise=true , address=145.79.227.18:4005 , piece_id=IMNWS75R4MTBLDUVYR6TCOWZLB5U3763R56HKKO3LSKGJVPSIGNQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.046904954 (566h23m45.539337399s) [noise=true , address=45.91.116.8:4045 , piece_id=WQBNA3NDZBQ6NAMLMB4IKGOZIH4SHZACNOSMRQBUJXMSO4AKESVA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047049339 (566h23m45.53948179s) [noise=true , address=45.91.116.11:4117 , piece_id=2ZNQUGE6R2QOZJCRA4KWT2Y7IKZ5ZPQWTKNTSMK3WOZ4KUN5KQZQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047469004 (566h23m45.539901457s) [noise=true , address=77.90.31.19:4841 , piece_id=ZFNAUEMLT6VWPVIP53EKINHKNWC4H7UNJSTL2OYUS6WPFAUOPS5A, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.047957742 (566h23m45.540390184s) [noise=true , address=77.90.31.12:4589 , piece_id=J2VRYHV43YRSOJ2X7NXRSO35O4JNBJELON27GFGJBSQG3L7DIKUA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048390500 (566h23m45.540822989s) [noise=true , address=145.79.227.17:4970 , piece_id=A5S2OJM5VUCSEZAJFOWHQK2XUKXUTT5VVSQLKK6ZC62ARU6WZSPQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048619410 (566h23m45.541051859s) [noise=true , address=77.90.31.15:4697 , piece_id=NBBDIDC64NAZKUYUGLHV5DAHDZKHHDHDWS3OBQG3VF5GOUHZVPWA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.048962659 (566h23m45.541395101s) [noise=true , address=45.91.116.7:4010 , piece_id=OWKSKTCLX3JFSDOCEP37IHTAO32TICL7F7YMSK6KIH47ZRGJIT5Q, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049294858 (566h23m45.541727303s) [noise=true , address=77.90.31.16:4734 , piece_id=6PPHPDJRMRKNEYHMDDZVCLICO5KUXMZAFBZ7EF6XVMYKGSPTVDSQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049592400 (566h23m45.542024845s) [noise=true , address=77.90.31.10:4520 , piece_id=AGNOJKORSH4CEZUJ2P6CFFHVNWK6IB2S5GXUSXGU4ALMPZJCFTQQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.049939474 (566h23m45.542371921s) [noise=true , address=145.79.227.17:4971 , piece_id=RWVET42YTHRDPI7D6GRSYUK7EIDRSWCZ5VNDIKCA4VRMGUCVBAFQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.050366023 (566h23m45.542798478s) [noise=true , address=45.91.116.14:4190 , piece_id=TS3M625BQNTBC4YBH3GOVRJ323CBRV34K3CAJMX7FNSQBYWSSOYQ, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.050850463 (566h23m45.543282914s) [noise=true , address=77.90.31.18:4805 , piece_id=F5IQFMGK7T5IGFALDPPMC4QBKMFUQ2OG7622QA2YJBUQDWBS4CVA, seg_pos={0 0}, upload=498269]: Uploading piece... Sep 22 04:31:25.667633076 (566h23m46.160065509s) [upload=498269]: Next returned nil segment Sep 22 04:31:25.667664989 (566h23m46.160097432s) [upload=498269]: Waiting for error group managing segments... Sep 22 04:31:25.803338183 (566h23m46.295770615s) [noise=true , address=45.91.116.8:4045 , piece_id=WQBNA3NDZBQ6NAMLMB4IKGOZIH4SHZACNOSMRQBUJXMSO4AKESVA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.805064032 (566h23m46.297496465s) [noise=true , address=77.90.31.10:4520 , piece_id=AGNOJKORSH4CEZUJ2P6CFFHVNWK6IB2S5GXUSXGU4ALMPZJCFTQQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.806116220 (566h23m46.298548651s) [noise=true , address=45.91.116.7:4010 , piece_id=OWKSKTCLX3JFSDOCEP37IHTAO32TICL7F7YMSK6KIH47ZRGJIT5Q, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.806176194 (566h23m46.298608727s) [noise=true , address=45.91.116.9:4261 , piece_id=CEGMAKDYVS3WEZVJLWZ7PLPIFF5HGU72XR3RYN5KFI6JYY6773MQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.809766126 (566h23m46.30219857s) [noise=true , address=45.91.116.14:4190 , piece_id=TS3M625BQNTBC4YBH3GOVRJ323CBRV34K3CAJMX7FNSQBYWSSOYQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.810019259 (566h23m46.302451696s) [noise=true , address=45.91.116.11:4117 , piece_id=2ZNQUGE6R2QOZJCRA4KWT2Y7IKZ5ZPQWTKNTSMK3WOZ4KUN5KQZQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.810642598 (566h23m46.303075034s) [noise=true , address=77.90.31.12:4589 , piece_id=J2VRYHV43YRSOJ2X7NXRSO35O4JNBJELON27GFGJBSQG3L7DIKUA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.811792355 (566h23m46.304224799s) [noise=true , address=77.90.31.19:4841 , piece_id=ZFNAUEMLT6VWPVIP53EKINHKNWC4H7UNJSTL2OYUS6WPFAUOPS5A, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.811940311 (566h23m46.304372744s) [noise=true , address=45.91.116.13:4155 , piece_id=7S37SPYHOEXLMC6QHLGZLH7KRXD7SCV3U4LQ62OBO2GOEBMY4DXA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.812546498 (566h23m46.304978934s) [noise=true , address=77.90.31.18:4805 , piece_id=F5IQFMGK7T5IGFALDPPMC4QBKMFUQ2OG7622QA2YJBUQDWBS4CVA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.813702226 (566h23m46.306134661s) [noise=true , address=77.90.31.16:4734 , piece_id=6PPHPDJRMRKNEYHMDDZVCLICO5KUXMZAFBZ7EF6XVMYKGSPTVDSQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:25.814617534 (566h23m46.307049965s) [noise=true , address=77.90.31.15:4697 , piece_id=NBBDIDC64NAZKUYUGLHV5DAHDZKHHDHDWS3OBQG3VF5GOUHZVPWA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: <nil> Sep 22 04:31:41.942437754 (566h24m2.43487019s) [noise=true , address=145.79.227.18:4005 , piece_id=IMNWS75R4MTBLDUVYR6TCOWZLB5U3763R56HKKO3LSKGJVPSIGNQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:31:54.607167993 (566h24m15.099600436s) [noise=true , address=145.79.227.17:4970 , piece_id=A5S2OJM5VUCSEZAJFOWHQK2XUKXUTT5VVSQLKK6ZC62ARU6WZSPQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607533226 (566h24m15.099965694s) [noise=true , address=145.79.227.17:4971 , piece_id=RWVET42YTHRDPI7D6GRSYUK7EIDRSWCZ5VNDIKCA4VRMGUCVBAFQ, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607775799 (566h24m15.100208249s) [noise=true , address=145.79.227.2:4115 , piece_id=4BO6TSTGW7MMJTLXNQFU55C7I2OBGPJZWQKW4R5U2SPAWQS6IHPA, seg_pos={0 0}, upload=498269]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:31:54.607801850 (566h24m15.100234304s) [seg_pos={0 0}, upload=498269]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:31:54.718884679 (566h24m15.211317115s) [upload=498269]: Done uploading segments... Sep 22 04:31:54.718934548 (566h24m15.211366981s) [upload=498269]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.11397948s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [....X..XXXX.........................XX..X..X......X.........] duration: 2.856296396s FAIL (2025-09-22 04:31:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.856338419s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.....X..XXXX.........................XX..X..X......X........] duration: 2.859118422s FAIL (2025-09-22 04:31:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.859156738s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [......X..XXXX.........................XX..X..X......X.......] duration: 2.751397488s FAIL (2025-09-22 04:30:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.751448005s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [....X..X..XX..X......XXX.....XX.X..X.XX..X..X.X.............] duration: 2.632285428s FAIL (2025-09-22 04:30:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:20:25.267832059 (566h12m45.760264491s) [upload=498115]: Starting upload Sep 22 04:20:25.267913906 (566h12m45.760346343s) [upload=498115]: Done starting upload Sep 22 04:20:25.268155840 (566h12m45.760588274s) [upload=498115]: Uploading segments... Sep 22 04:20:25.268434238 (566h12m45.760866674s) [upload=498115]: Got next segment. Inline: false Sep 22 04:20:25.268526154 (566h12m45.760958589s) [upload=498115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:20:25.268533661 (566h12m45.760966124s) [upload=498115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:20:25.385146545 (566h12m45.877578986s) [seg_pos={0 0}, upload=498115]: Begin upload segment... Sep 22 04:20:25.385197224 (566h12m45.877629653s) [seg_pos={0 0}, upload=498115]: Done begin upload segment. Sep 22 04:20:25.385225061 (566h12m45.877657507s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385259555 (566h12m45.877692007s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385580251 (566h12m45.878012708s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385961294 (566h12m45.87839374s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386422953 (566h12m45.878855402s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386492567 (566h12m45.878925021s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386825686 (566h12m45.87925813s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387102239 (566h12m45.879534684s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387404982 (566h12m45.879837429s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387540348 (566h12m45.879972793s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387949155 (566h12m45.880381609s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388326089 (566h12m45.880758553s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388663859 (566h12m45.881096311s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388753731 (566h12m45.881186179s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389081946 (566h12m45.881514389s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389379226 (566h12m45.881811669s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.990863597 (566h12m46.483296025s) [upload=498115]: Next returned nil segment Sep 22 04:20:25.990881639 (566h12m46.48331408s) [upload=498115]: Waiting for error group managing segments... Sep 22 04:20:26.130614295 (566h12m46.623046727s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132016117 (566h12m46.624448549s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132373859 (566h12m46.624806303s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132531076 (566h12m46.624963504s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.133649560 (566h12m46.626081992s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.134739518 (566h12m46.627171955s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.135235002 (566h12m46.627667438s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.139057204 (566h12m46.631489636s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.150084683 (566h12m46.642517117s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.153229156 (566h12m46.645661591s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156966637 (566h12m46.649399069s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156982994 (566h12m46.649415423s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:54.962276597 (566h13m15.454709039s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:20:54.962491300 (566h13m15.45492375s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 04:20:54.962561914 (566h13m15.454994349s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; context deadline exceeded Sep 22 04:20:54.962625206 (566h13m15.455057652s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:20:54.962699426 (566h13m15.455131877s) [seg_pos={0 0}, upload=498115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:20:55.071381920 (566h13m15.563814385s) [upload=498115]: Done uploading segments... Sep 22 04:20:55.071433178 (566h13m15.563865628s) [upload=498115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.632383437s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [X......X..XXXX.........................XX..X..X......X......] duration: 2.724495917s FAIL (2025-09-22 04:30:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.724547727s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [.....X..X..XX..X......XXX.....XX.X..X.XX..X..X.X............] duration: 2.517414362s FAIL (2025-09-22 04:29:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:20:25.267832059 (566h12m45.760264491s) [upload=498115]: Starting upload Sep 22 04:20:25.267913906 (566h12m45.760346343s) [upload=498115]: Done starting upload Sep 22 04:20:25.268155840 (566h12m45.760588274s) [upload=498115]: Uploading segments... Sep 22 04:20:25.268434238 (566h12m45.760866674s) [upload=498115]: Got next segment. Inline: false Sep 22 04:20:25.268526154 (566h12m45.760958589s) [upload=498115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:20:25.268533661 (566h12m45.760966124s) [upload=498115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:20:25.385146545 (566h12m45.877578986s) [seg_pos={0 0}, upload=498115]: Begin upload segment... Sep 22 04:20:25.385197224 (566h12m45.877629653s) [seg_pos={0 0}, upload=498115]: Done begin upload segment. Sep 22 04:20:25.385225061 (566h12m45.877657507s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385259555 (566h12m45.877692007s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385580251 (566h12m45.878012708s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385961294 (566h12m45.87839374s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386422953 (566h12m45.878855402s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386492567 (566h12m45.878925021s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386825686 (566h12m45.87925813s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387102239 (566h12m45.879534684s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387404982 (566h12m45.879837429s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387540348 (566h12m45.879972793s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387949155 (566h12m45.880381609s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388326089 (566h12m45.880758553s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388663859 (566h12m45.881096311s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388753731 (566h12m45.881186179s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389081946 (566h12m45.881514389s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389379226 (566h12m45.881811669s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.990863597 (566h12m46.483296025s) [upload=498115]: Next returned nil segment Sep 22 04:20:25.990881639 (566h12m46.48331408s) [upload=498115]: Waiting for error group managing segments... Sep 22 04:20:26.130614295 (566h12m46.623046727s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132016117 (566h12m46.624448549s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132373859 (566h12m46.624806303s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132531076 (566h12m46.624963504s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.133649560 (566h12m46.626081992s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.134739518 (566h12m46.627171955s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.135235002 (566h12m46.627667438s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.139057204 (566h12m46.631489636s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.150084683 (566h12m46.642517117s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.153229156 (566h12m46.645661591s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156966637 (566h12m46.649399069s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156982994 (566h12m46.649415423s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:54.962276597 (566h13m15.454709039s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:20:54.962491300 (566h13m15.45492375s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 04:20:54.962561914 (566h13m15.454994349s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; context deadline exceeded Sep 22 04:20:54.962625206 (566h13m15.455057652s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:20:54.962699426 (566h13m15.455131877s) [seg_pos={0 0}, upload=498115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:20:55.071381920 (566h13m15.563814385s) [upload=498115]: Done uploading segments... Sep 22 04:20:55.071433178 (566h13m15.563865628s) [upload=498115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.517456869s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.X......X..XXXX.........................XX..X..X......X.....] duration: 2.911855208s FAIL (2025-09-22 04:29:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.911885545s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [......X..X..XX..X......XXX.....XX.X..X.XX..X..X.X...........] duration: 2.557522002s FAIL (2025-09-22 04:29:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:20:25.267832059 (566h12m45.760264491s) [upload=498115]: Starting upload Sep 22 04:20:25.267913906 (566h12m45.760346343s) [upload=498115]: Done starting upload Sep 22 04:20:25.268155840 (566h12m45.760588274s) [upload=498115]: Uploading segments... Sep 22 04:20:25.268434238 (566h12m45.760866674s) [upload=498115]: Got next segment. Inline: false Sep 22 04:20:25.268526154 (566h12m45.760958589s) [upload=498115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:20:25.268533661 (566h12m45.760966124s) [upload=498115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:20:25.385146545 (566h12m45.877578986s) [seg_pos={0 0}, upload=498115]: Begin upload segment... Sep 22 04:20:25.385197224 (566h12m45.877629653s) [seg_pos={0 0}, upload=498115]: Done begin upload segment. Sep 22 04:20:25.385225061 (566h12m45.877657507s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385259555 (566h12m45.877692007s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385580251 (566h12m45.878012708s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385961294 (566h12m45.87839374s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386422953 (566h12m45.878855402s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386492567 (566h12m45.878925021s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386825686 (566h12m45.87925813s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387102239 (566h12m45.879534684s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387404982 (566h12m45.879837429s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387540348 (566h12m45.879972793s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387949155 (566h12m45.880381609s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388326089 (566h12m45.880758553s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388663859 (566h12m45.881096311s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388753731 (566h12m45.881186179s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389081946 (566h12m45.881514389s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389379226 (566h12m45.881811669s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.990863597 (566h12m46.483296025s) [upload=498115]: Next returned nil segment Sep 22 04:20:25.990881639 (566h12m46.48331408s) [upload=498115]: Waiting for error group managing segments... Sep 22 04:20:26.130614295 (566h12m46.623046727s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132016117 (566h12m46.624448549s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132373859 (566h12m46.624806303s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132531076 (566h12m46.624963504s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.133649560 (566h12m46.626081992s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.134739518 (566h12m46.627171955s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.135235002 (566h12m46.627667438s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.139057204 (566h12m46.631489636s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.150084683 (566h12m46.642517117s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.153229156 (566h12m46.645661591s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156966637 (566h12m46.649399069s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156982994 (566h12m46.649415423s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:54.962276597 (566h13m15.454709039s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:20:54.962491300 (566h13m15.45492375s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 04:20:54.962561914 (566h13m15.454994349s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; context deadline exceeded Sep 22 04:20:54.962625206 (566h13m15.455057652s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:20:54.962699426 (566h13m15.455131877s) [seg_pos={0 0}, upload=498115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:20:55.071381920 (566h13m15.563814385s) [upload=498115]: Done uploading segments... Sep 22 04:20:55.071433178 (566h13m15.563865628s) [upload=498115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.557560671s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [.......X..X..XX..X......XXX.....XX.X..X.XX..X..X.X..........] duration: 2.530230463s FAIL (2025-09-22 04:28:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:20:25.267832059 (566h12m45.760264491s) [upload=498115]: Starting upload Sep 22 04:20:25.267913906 (566h12m45.760346343s) [upload=498115]: Done starting upload Sep 22 04:20:25.268155840 (566h12m45.760588274s) [upload=498115]: Uploading segments... Sep 22 04:20:25.268434238 (566h12m45.760866674s) [upload=498115]: Got next segment. Inline: false Sep 22 04:20:25.268526154 (566h12m45.760958589s) [upload=498115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:20:25.268533661 (566h12m45.760966124s) [upload=498115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:20:25.385146545 (566h12m45.877578986s) [seg_pos={0 0}, upload=498115]: Begin upload segment... Sep 22 04:20:25.385197224 (566h12m45.877629653s) [seg_pos={0 0}, upload=498115]: Done begin upload segment. Sep 22 04:20:25.385225061 (566h12m45.877657507s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385259555 (566h12m45.877692007s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385580251 (566h12m45.878012708s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385961294 (566h12m45.87839374s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386422953 (566h12m45.878855402s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386492567 (566h12m45.878925021s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386825686 (566h12m45.87925813s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387102239 (566h12m45.879534684s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387404982 (566h12m45.879837429s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387540348 (566h12m45.879972793s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387949155 (566h12m45.880381609s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388326089 (566h12m45.880758553s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388663859 (566h12m45.881096311s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388753731 (566h12m45.881186179s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389081946 (566h12m45.881514389s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389379226 (566h12m45.881811669s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.990863597 (566h12m46.483296025s) [upload=498115]: Next returned nil segment Sep 22 04:20:25.990881639 (566h12m46.48331408s) [upload=498115]: Waiting for error group managing segments... Sep 22 04:20:26.130614295 (566h12m46.623046727s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132016117 (566h12m46.624448549s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132373859 (566h12m46.624806303s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132531076 (566h12m46.624963504s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.133649560 (566h12m46.626081992s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.134739518 (566h12m46.627171955s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.135235002 (566h12m46.627667438s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.139057204 (566h12m46.631489636s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.150084683 (566h12m46.642517117s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.153229156 (566h12m46.645661591s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156966637 (566h12m46.649399069s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156982994 (566h12m46.649415423s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:54.962276597 (566h13m15.454709039s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:20:54.962491300 (566h13m15.45492375s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 04:20:54.962561914 (566h13m15.454994349s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; context deadline exceeded Sep 22 04:20:54.962625206 (566h13m15.455057652s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:20:54.962699426 (566h13m15.455131877s) [seg_pos={0 0}, upload=498115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:20:55.071381920 (566h13m15.563814385s) [upload=498115]: Done uploading segments... Sep 22 04:20:55.071433178 (566h13m15.563865628s) [upload=498115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.530387767s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..X......X..XXXX.........................XX..X..X......X....] duration: 2.897802555s FAIL (2025-09-22 04:28:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.897870145s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [........X..X..XX..X......XXX.....XX.X..X.XX..X..X.X.........] duration: 2.636410586s FAIL (2025-09-22 04:27:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:20:25.267832059 (566h12m45.760264491s) [upload=498115]: Starting upload Sep 22 04:20:25.267913906 (566h12m45.760346343s) [upload=498115]: Done starting upload Sep 22 04:20:25.268155840 (566h12m45.760588274s) [upload=498115]: Uploading segments... Sep 22 04:20:25.268434238 (566h12m45.760866674s) [upload=498115]: Got next segment. Inline: false Sep 22 04:20:25.268526154 (566h12m45.760958589s) [upload=498115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:20:25.268533661 (566h12m45.760966124s) [upload=498115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:20:25.385146545 (566h12m45.877578986s) [seg_pos={0 0}, upload=498115]: Begin upload segment... Sep 22 04:20:25.385197224 (566h12m45.877629653s) [seg_pos={0 0}, upload=498115]: Done begin upload segment. Sep 22 04:20:25.385225061 (566h12m45.877657507s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385259555 (566h12m45.877692007s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385580251 (566h12m45.878012708s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385961294 (566h12m45.87839374s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386422953 (566h12m45.878855402s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386492567 (566h12m45.878925021s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386825686 (566h12m45.87925813s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387102239 (566h12m45.879534684s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387404982 (566h12m45.879837429s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387540348 (566h12m45.879972793s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387949155 (566h12m45.880381609s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388326089 (566h12m45.880758553s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388663859 (566h12m45.881096311s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388753731 (566h12m45.881186179s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389081946 (566h12m45.881514389s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389379226 (566h12m45.881811669s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.990863597 (566h12m46.483296025s) [upload=498115]: Next returned nil segment Sep 22 04:20:25.990881639 (566h12m46.48331408s) [upload=498115]: Waiting for error group managing segments... Sep 22 04:20:26.130614295 (566h12m46.623046727s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132016117 (566h12m46.624448549s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132373859 (566h12m46.624806303s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132531076 (566h12m46.624963504s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.133649560 (566h12m46.626081992s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.134739518 (566h12m46.627171955s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.135235002 (566h12m46.627667438s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.139057204 (566h12m46.631489636s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.150084683 (566h12m46.642517117s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.153229156 (566h12m46.645661591s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156966637 (566h12m46.649399069s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156982994 (566h12m46.649415423s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:54.962276597 (566h13m15.454709039s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:20:54.962491300 (566h13m15.45492375s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 04:20:54.962561914 (566h13m15.454994349s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; context deadline exceeded Sep 22 04:20:54.962625206 (566h13m15.455057652s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:20:54.962699426 (566h13m15.455131877s) [seg_pos={0 0}, upload=498115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:20:55.071381920 (566h13m15.563814385s) [upload=498115]: Done uploading segments... Sep 22 04:20:55.071433178 (566h13m15.563865628s) [upload=498115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.636450886s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...X......X..XXXX.........................XX..X..X......X...] duration: 2.733395625s FAIL (2025-09-22 04:27:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.733512082s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [.........X..X..XX..X......XXX.....XX.X..X.XX..X..X.X........] duration: 2.370647049s FAIL (2025-09-22 04:27:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:20:25.267832059 (566h12m45.760264491s) [upload=498115]: Starting upload Sep 22 04:20:25.267913906 (566h12m45.760346343s) [upload=498115]: Done starting upload Sep 22 04:20:25.268155840 (566h12m45.760588274s) [upload=498115]: Uploading segments... Sep 22 04:20:25.268434238 (566h12m45.760866674s) [upload=498115]: Got next segment. Inline: false Sep 22 04:20:25.268526154 (566h12m45.760958589s) [upload=498115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:20:25.268533661 (566h12m45.760966124s) [upload=498115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:20:25.385146545 (566h12m45.877578986s) [seg_pos={0 0}, upload=498115]: Begin upload segment... Sep 22 04:20:25.385197224 (566h12m45.877629653s) [seg_pos={0 0}, upload=498115]: Done begin upload segment. Sep 22 04:20:25.385225061 (566h12m45.877657507s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385259555 (566h12m45.877692007s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385580251 (566h12m45.878012708s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385961294 (566h12m45.87839374s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386422953 (566h12m45.878855402s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386492567 (566h12m45.878925021s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386825686 (566h12m45.87925813s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387102239 (566h12m45.879534684s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387404982 (566h12m45.879837429s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387540348 (566h12m45.879972793s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387949155 (566h12m45.880381609s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388326089 (566h12m45.880758553s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388663859 (566h12m45.881096311s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388753731 (566h12m45.881186179s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389081946 (566h12m45.881514389s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389379226 (566h12m45.881811669s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.990863597 (566h12m46.483296025s) [upload=498115]: Next returned nil segment Sep 22 04:20:25.990881639 (566h12m46.48331408s) [upload=498115]: Waiting for error group managing segments... Sep 22 04:20:26.130614295 (566h12m46.623046727s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132016117 (566h12m46.624448549s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132373859 (566h12m46.624806303s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132531076 (566h12m46.624963504s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.133649560 (566h12m46.626081992s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.134739518 (566h12m46.627171955s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.135235002 (566h12m46.627667438s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.139057204 (566h12m46.631489636s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.150084683 (566h12m46.642517117s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.153229156 (566h12m46.645661591s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156966637 (566h12m46.649399069s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156982994 (566h12m46.649415423s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:54.962276597 (566h13m15.454709039s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:20:54.962491300 (566h13m15.45492375s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 04:20:54.962561914 (566h13m15.454994349s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; context deadline exceeded Sep 22 04:20:54.962625206 (566h13m15.455057652s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:20:54.962699426 (566h13m15.455131877s) [seg_pos={0 0}, upload=498115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:20:55.071381920 (566h13m15.563814385s) [upload=498115]: Done uploading segments... Sep 22 04:20:55.071433178 (566h13m15.563865628s) [upload=498115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.370679979s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [....X......X..XXXX.........................XX..X..X......X..] duration: 2.977321698s FAIL (2025-09-22 04:27:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.977359868s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.....X......X..XXXX.........................XX..X..X......X.] duration: 2.786167103s FAIL (2025-09-22 04:26:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.78619362s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)