Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [X......X..............X.X........X.XX......X....X...........] duration: 2.797983878s FAIL (2025-09-22 05:01:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.798007567s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.X......X..............X.X........X.XX......X....X..........] duration: 2.840474103s FAIL (2025-09-22 05:00:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.840514003s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..X.X.............X........X.X..............................] duration: 3.044964537s FAIL (2025-09-22 05:00:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 3.045004835s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..X......X..............X.X........X.XX......X....X.........] duration: 2.758673799s FAIL (2025-09-22 04:59:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.758753298s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [X..X......X..............X.X........X.XX......X....X........] duration: 2.773444654s FAIL (2025-09-22 04:59:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.77347279s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [X..X.X.............X........X.X.............................] duration: 2.462224847s FAIL (2025-09-22 04:59:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.46226015s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.X..X......X..............X.X........X.XX......X....X.......] duration: 2.788789449s FAIL (2025-09-22 04:58:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.788892852s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X..X.X.............X........X.X............................] duration: 2.575968887s FAIL (2025-09-22 04:58:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.576010493s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..X..X......X..............X.X........X.XX......X....X......] duration: 2.909117357s FAIL (2025-09-22 04:57:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.909142577s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..X..X.X.............X........X.X...........................] duration: 2.476532945s FAIL (2025-09-22 04:56:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.476560978s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [X..X..X......X..............X.X........X.XX......X....X.....] duration: 2.808292715s FAIL (2025-09-22 04:56:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.80850847s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [X..X..X.X.............X........X.X..........................] duration: 2.67469249s FAIL (2025-09-22 04:56:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.674791864s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [XX..X..X......X..............X.X........X.XX......X....X....] duration: 3.029463051s FAIL (2025-09-22 04:56:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.030689934s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.XX..X..X......X..............X.X........X.XX......X....X...] duration: 2.861162694s FAIL (2025-09-22 04:55:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.861194169s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [XX..X..X.X.............X........X.X.........................] duration: 2.392038218s FAIL (2025-09-22 04:55:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.392074599s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.XX..X..X.X.............X........X.X........................] duration: 2.718583032s FAIL (2025-09-22 04:55:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.71861412s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [..XX..X..X......X..............X.X........X.XX......X....X..] duration: 2.797998961s FAIL (2025-09-22 04:54:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.798027001s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [...XX..X..X......X..............X.X........X.XX......X....X.] duration: 2.937543799s FAIL (2025-09-22 04:54:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.93757301s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [X.XX..X..X.X.............X........X.X.......................] duration: 2.762412989s FAIL (2025-09-22 04:54:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.762447687s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [....XX..X..X......X..............X.X........X.XX......X....X] duration: 30.001593306s FAIL (2025-09-22 04:54:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001619002s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.....XX..X..X......X..............X.X........X.XX......X....] duration: 2.615801174s FAIL (2025-09-22 04:53:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.615832898s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.X.XX..X..X.X.............X........X.X......................] duration: 2.408396329s FAIL (2025-09-22 04:53:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.408431006s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [......XX..X..X......X..............X.X........X.XX......X...] duration: 2.723725837s FAIL (2025-09-22 04:52:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.72375507s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..X.XX..X..X.X.............X........X.X.....................] duration: 2.452066709s FAIL (2025-09-22 04:52:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.452133584s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.......XX..X..X......X..............X.X........X.XX......X..] duration: 2.928435033s FAIL (2025-09-22 04:52:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.92846523s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [X..X.XX..X..X.X.............X........X.X....................] duration: 2.798835244s FAIL (2025-09-22 04:52:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.798868034s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [........XX..X..X......X..............X.X........X.XX......X.] duration: 2.686545477s FAIL (2025-09-22 04:51:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.686578558s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.X..X.XX..X..X.X.............X........X.X...................] duration: 2.615479939s FAIL (2025-09-22 04:51:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.615515656s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [X.X..X.XX..X..X.X.............X........X.X..................] duration: 2.592621683s FAIL (2025-09-22 04:51:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.592658972s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.........XX..X..X......X..............X.X........X.XX......X] duration: 30.000618666s FAIL (2025-09-22 04:51:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000644229s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [XX.X..X.XX..X..X.X.............X........X.X.................] duration: 2.487989146s FAIL (2025-09-22 04:50:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.48801913s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.XX.X..X.XX..X..X.X.............X........X.X................] duration: 2.540627115s FAIL (2025-09-22 04:49:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.540659149s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..........XX..X..X......X..............X.X........X.XX......] duration: 2.831153235s FAIL (2025-09-22 04:49:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.8311811s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...........XX..X..X......X..............X.X........X.XX.....] duration: 2.683554144s FAIL (2025-09-22 04:48:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.683591341s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [..XX.X..X.XX..X..X.X.............X........X.X...............] duration: 2.503070607s FAIL (2025-09-22 04:48:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.503101254s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [............XX..X..X......X..............X.X........X.XX....] duration: 2.973731382s FAIL (2025-09-22 04:48:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.973795499s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [...XX.X..X.XX..X..X.X.............X........X.X..............] duration: 2.651231457s FAIL (2025-09-22 04:47:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.651261299s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.............XX..X..X......X..............X.X........X.XX...] duration: 2.91121614s FAIL (2025-09-22 04:47:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.911241797s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [....XX.X..X.XX..X..X.X.............X........X.X.............] duration: 2.633218141s FAIL (2025-09-22 04:46:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.633245835s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..............XX..X..X......X..............X.X........X.XX..] duration: 2.904740221s FAIL (2025-09-22 04:46:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.904767237s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.....XX.X..X.XX..X..X.X.............X........X.X............] duration: 2.720950174s FAIL (2025-09-22 04:46:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.72098758s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...............XX..X..X......X..............X.X........X.XX.] duration: 2.663883869s FAIL (2025-09-22 04:46:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.663913304s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [X.....XX.X..X.XX..X..X.X.............X........X.X...........] duration: 2.442230063s FAIL (2025-09-22 04:45:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.442267062s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [................XX..X..X......X..............X.X........X.XX] duration: 30.000573619s FAIL (2025-09-22 04:45:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000614238s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.................XX..X..X......X..............X.X........X.X] duration: 30.000869794s FAIL (2025-09-22 04:44:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000896118s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [XX.....XX.X..X.XX..X..X.X.............X........X.X..........] duration: 2.395343822s FAIL (2025-09-22 04:44:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.395380797s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [XXX.....XX.X..X.XX..X..X.X.............X........X.X.........] duration: 2.420882956s FAIL (2025-09-22 04:43:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.420914926s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..................XX..X..X......X..............X.X........X.] duration: 2.611031619s FAIL (2025-09-22 04:43:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.61146783s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...................XX..X..X......X..............X.X........X] duration: 30.001572753s FAIL (2025-09-22 04:43:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001667664s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.XXX.....XX.X..X.XX..X..X.X.............X........X.X........] duration: 2.565749241s FAIL (2025-09-22 04:42:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:37:32.968419254 (566h29m53.460851689s) [upload=498358]: Starting upload Sep 22 04:37:32.968513489 (566h29m53.460945923s) [upload=498358]: Done starting upload Sep 22 04:37:32.968809636 (566h29m53.461242073s) [upload=498358]: Uploading segments... Sep 22 04:37:32.969040319 (566h29m53.461472764s) [upload=498358]: Got next segment. Inline: false Sep 22 04:37:32.969163301 (566h29m53.461595748s) [upload=498358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:37:32.969172925 (566h29m53.461605372s) [upload=498358]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:37:33.080710082 (566h29m53.573142539s) [seg_pos={0 0}, upload=498358]: Begin upload segment... Sep 22 04:37:33.080779723 (566h29m53.573212156s) [seg_pos={0 0}, upload=498358]: Done begin upload segment. Sep 22 04:37:33.080813398 (566h29m53.573245854s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.080845445 (566h29m53.57327789s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081381456 (566h29m53.573813942s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081398382 (566h29m53.573830837s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081837215 (566h29m53.574269729s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.081945290 (566h29m53.574377735s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.082577692 (566h29m53.57501014s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086324936 (566h29m53.578757379s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.086657904 (566h29m53.579090358s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087163160 (566h29m53.579595606s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.087616272 (566h29m53.580048714s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088013469 (566h29m53.580445911s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088434498 (566h29m53.580866944s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.088856358 (566h29m53.58128881s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089265407 (566h29m53.581697864s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.089873098 (566h29m53.582305553s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Uploading piece... Sep 22 04:37:33.708120441 (566h29m54.200552898s) [upload=498358]: Next returned nil segment Sep 22 04:37:33.708153651 (566h29m54.200586107s) [upload=498358]: Waiting for error group managing segments... Sep 22 04:37:33.839160529 (566h29m54.331592974s) [noise=true , address=45.91.116.13:4155 , piece_id=Y3Y5EPNU34SK7ORARWWYOB7N56U43MFLH5CZOIVJONJ2HSHKKJLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.839427891 (566h29m54.331860345s) [noise=true , address=77.90.31.18:4807 , piece_id=LPOTW6ZJJULK3OA6XOTP2KXZOBYFXS34M2ZAHLWGP2MJA2JB4HLQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.846774756 (566h29m54.33920721s) [noise=true , address=77.90.31.11:4553 , piece_id=O2EWRCDMBKGRY2FDJDHM7F5OF555SBX55FM76B7NTC2BHS5DGEFQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.847735851 (566h29m54.340168295s) [noise=true , address=45.91.116.4:4940 , piece_id=UXIF6WUZ2PJIKMPWY7TIQ3VKZXARAVN53HCSXYUI635EOWOWKFHQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.848490229 (566h29m54.340922678s) [noise=true , address=45.91.116.5:4299 , piece_id=25UQC76R26MMVM5FTIIZ54UYQFIZFRGFD5CMCAEVWZOUVHZG4DMQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849277942 (566h29m54.341710379s) [noise=true , address=45.91.116.4:4937 , piece_id=T5EIJ4LVXH2YAY7HFFEF5YKWSXPVUK2BD6VCXAYVSWBH7IU7YH3Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.849971905 (566h29m54.342404352s) [noise=true , address=45.91.116.14:4191 , piece_id=FSNAAUR6PL7P7EFLFF6HIDZE2BTMZZCW7PSQYO2CVUN7XFCKSGJQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.850312853 (566h29m54.342745289s) [noise=true , address=45.91.116.6:4975 , piece_id=FLFCY5VZG4MTLRNMJHYWJEE3KVSHXH3JJCH5OAQ3YK6FI53MLRYA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.857758135 (566h29m54.350190579s) [noise=true , address=45.91.116.13:4153 , piece_id=TBGNBL43AWH3P6WUZBBTMAEGML5INHON47ELYNTBRMVISN5IM7ZQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.859398765 (566h29m54.351831211s) [noise=true , address=77.90.31.13:4627 , piece_id=7MO6XXZAXKVBRVCCWHOXRWZSZQXNRBXJVWBBLZBDCTMPYJDYY6KQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.862640583 (566h29m54.35507303s) [noise=true , address=45.91.116.9:4261 , piece_id=ONNMC5GGO2KY5VQKFAO4IDCQC5W4EYYHIZBXPTG5EFIFQJ5KSTVQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:33.865251417 (566h29m54.357683864s) [noise=true , address=45.91.116.10:4081 , piece_id=Z3V3FFVACDB6JFLKZS24YXY5A2AJ7HPIAWKETEGWK4QOT5SNCBGQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: <nil> Sep 22 04:37:35.575020212 (566h29m56.067452669s) [noise=true , address=145.79.227.18:4005 , piece_id=N6EUH4XX5UTTT7PY5OJ33PGXUFXJ6J4OUNEOMBK7HQ3GWKU6GNCA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:38:02.694641552 (566h30m23.187073992s) [noise=true , address=145.79.227.7:4293 , piece_id=BEVQOKI2OZQSY3HIA4CNISMNCBEGY24UJUVMHQGSZ5ADWAHWSROQ, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694933020 (566h30m23.18736546s) [noise=true , address=145.79.227.13:4804 , piece_id=633SWUJBABGZ6BB2X7SF4KERVZUZ3XPDOLK4TQPI7WRCNUP27X6Q, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694953564 (566h30m23.18738602s) [noise=true , address=145.79.227.4:4187 , piece_id=UXPTQXBW64WFA4NU6UBX27ORZX2G42MT4LYTCJZIHKH554EE4PLA, seg_pos={0 0}, upload=498358]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:38:02.694998543 (566h30m23.187430994s) [seg_pos={0 0}, upload=498358]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:38:02.800453269 (566h30m23.292885745s) [upload=498358]: Done uploading segments... Sep 22 04:38:02.800523687 (566h30m23.292956145s) [upload=498358]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.565798167s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)