Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................XX...........................] duration: 2.727408695s FAIL (2025-08-17 03:52:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.727432854s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [X...................X........................X..............] duration: 2.403061222s FAIL (2025-08-17 03:52:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.403094357s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................XX..........................] duration: 3.000718993s FAIL (2025-08-17 03:51:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.000766276s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X...................X........................X.............] duration: 2.565509266s FAIL (2025-08-17 03:51:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.56553769s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................XX.........................] duration: 2.732140758s FAIL (2025-08-17 03:50:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.732165529s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................XX........................] duration: 2.572234812s FAIL (2025-08-17 03:50:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.572259912s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X...................X........................X............] duration: 2.486757119s FAIL (2025-08-17 03:50:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.486796673s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................XX.......................] duration: 2.593354714s FAIL (2025-08-17 03:49:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.59338376s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X...................X........................X...........] duration: 2.357985328s FAIL (2025-08-17 03:49:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.358014103s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................XX......................] duration: 2.924495509s FAIL (2025-08-17 03:49:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.924524634s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................XX.....................] duration: 2.737246251s FAIL (2025-08-17 03:48:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.737277725s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X...................X........................X..........] duration: 2.381414162s FAIL (2025-08-17 03:48:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.381451164s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................XX....................] duration: 2.646219458s FAIL (2025-08-17 03:47:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.646258867s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X...................X........................X.........] duration: 2.607642181s FAIL (2025-08-17 03:46:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.607696458s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................XX...................] duration: 2.82072449s FAIL (2025-08-17 03:46:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.820759622s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X...................X........................X........] duration: 2.338925602s FAIL (2025-08-17 03:46:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.338953454s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X...................X........................X.......] duration: 2.392230906s FAIL (2025-08-17 03:45:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.392279916s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................XX..................] duration: 2.877023627s FAIL (2025-08-17 03:45:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.877057606s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X...................X........................X......] duration: 2.404990933s FAIL (2025-08-17 03:44:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.40508485s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................XX.................] duration: 2.840587164s FAIL (2025-08-17 03:44:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.840656447s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X...................X........................X.....] duration: 2.642546153s FAIL (2025-08-17 03:43:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.642582643s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................XX................] duration: 5.130121166s FAIL (2025-08-17 03:43:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.130215158s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [X.........X...................X........................X....] duration: 2.454444117s FAIL (2025-08-17 03:42:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.454477644s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................XX...............] duration: 2.864385891s FAIL (2025-08-17 03:42:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.864411835s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X.........X...................X........................X...] duration: 2.365548338s FAIL (2025-08-17 03:41:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.365580811s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 866.762557ms FAIL (2025-08-17 03:41:39 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 02:58:16.481213600 (201h32m38.440310832s) [upload=209799]: Starting upload Aug 17 02:58:16.481294851 (201h32m38.440392083s) [upload=209799]: Done starting upload Aug 17 02:58:16.481388248 (201h32m38.440485481s) [upload=209799]: Uploading segments... Aug 17 02:58:16.481461866 (201h32m38.44055912s) [upload=209799]: Got next segment. Inline: true Aug 17 02:58:16.481467741 (201h32m38.440564988s) [upload=209799]: Waiting for error group managing segments... Aug 17 02:58:16.481531233 (201h32m38.440628465s) [upload=209799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 02:58:16.481537443 (201h32m38.44063469s) [upload=209799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 02:58:16.481541360 (201h32m38.440638606s) [upload=209799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 02:58:16.651015934 (201h32m38.610113181s) [upload=209799]: Done uploading segments... Aug 17 02:58:16.651037402 (201h32m38.610134641s) [upload=209799]: Upload finished. err: <nil> Duration:* 866.795746ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..X.........X...................X........................X..] duration: 2.400465261s FAIL (2025-08-17 03:41:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.400496592s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................................XX..............] duration: 2.771480073s FAIL (2025-08-17 03:41:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.77156585s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 873.657126ms FAIL (2025-08-17 03:41:11 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 02:58:16.481213600 (201h32m38.440310832s) [upload=209799]: Starting upload Aug 17 02:58:16.481294851 (201h32m38.440392083s) [upload=209799]: Done starting upload Aug 17 02:58:16.481388248 (201h32m38.440485481s) [upload=209799]: Uploading segments... Aug 17 02:58:16.481461866 (201h32m38.44055912s) [upload=209799]: Got next segment. Inline: true Aug 17 02:58:16.481467741 (201h32m38.440564988s) [upload=209799]: Waiting for error group managing segments... Aug 17 02:58:16.481531233 (201h32m38.440628465s) [upload=209799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 02:58:16.481537443 (201h32m38.44063469s) [upload=209799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 02:58:16.481541360 (201h32m38.440638606s) [upload=209799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 02:58:16.651015934 (201h32m38.610113181s) [upload=209799]: Done uploading segments... Aug 17 02:58:16.651037402 (201h32m38.610134641s) [upload=209799]: Upload finished. err: <nil> Duration:* 873.776832ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...X.........X...................X........................X.] duration: 2.453730464s FAIL (2025-08-17 03:41:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.453761045s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................................XX.............] duration: 2.667447182s FAIL (2025-08-17 03:40:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.667686282s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 945.351607ms FAIL (2025-08-17 03:40:35 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 02:58:16.481213600 (201h32m38.440310832s) [upload=209799]: Starting upload Aug 17 02:58:16.481294851 (201h32m38.440392083s) [upload=209799]: Done starting upload Aug 17 02:58:16.481388248 (201h32m38.440485481s) [upload=209799]: Uploading segments... Aug 17 02:58:16.481461866 (201h32m38.44055912s) [upload=209799]: Got next segment. Inline: true Aug 17 02:58:16.481467741 (201h32m38.440564988s) [upload=209799]: Waiting for error group managing segments... Aug 17 02:58:16.481531233 (201h32m38.440628465s) [upload=209799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 02:58:16.481537443 (201h32m38.44063469s) [upload=209799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 02:58:16.481541360 (201h32m38.440638606s) [upload=209799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 02:58:16.651015934 (201h32m38.610113181s) [upload=209799]: Done uploading segments... Aug 17 02:58:16.651037402 (201h32m38.610134641s) [upload=209799]: Upload finished. err: <nil> Duration:* 945.383655ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [....X.........X...................X........................X] duration: 2.402067873s FAIL (2025-08-17 03:40:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0fe20535773fb327f0e1285fd8d60c2bacd0f393282f9562a7724de9fba7434c != before d861b10cb2a70d29d52a521a7de003da28a3c1fb9f6ac2aa0f41e49f8d58fc1d ======LOGS====== Aug 17 03:40:21.175138804 (202h14m43.134236052s) [upload=210504]: Starting upload Aug 17 03:40:21.175208980 (202h14m43.134306217s) [upload=210504]: Done starting upload Aug 17 03:40:21.175379812 (202h14m43.134477046s) [upload=210504]: Uploading segments... Aug 17 03:40:21.175510502 (202h14m43.134607736s) [upload=210504]: Got next segment. Inline: false Aug 17 03:40:21.175587606 (202h14m43.134684852s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:40:21.175594399 (202h14m43.134691635s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:40:21.373030471 (202h14m43.332127716s) [seg_pos={0 0}, upload=210504]: Begin upload segment... Aug 17 03:40:21.373095598 (202h14m43.332192844s) [seg_pos={0 0}, upload=210504]: Done begin upload segment. Aug 17 03:40:21.373132912 (202h14m43.332230173s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.373186791 (202h14m43.332284067s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.374685881 (202h14m43.333783136s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376635904 (202h14m43.335733152s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.376973502 (202h14m43.336070751s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377215392 (202h14m43.336312647s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377509865 (202h14m43.336607118s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.377978724 (202h14m43.337075975s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378693200 (202h14m43.337790452s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.378774753 (202h14m43.337872008s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.379242861 (202h14m43.338340116s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.380069773 (202h14m43.339167024s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.381076965 (202h14m43.34017422s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.382172018 (202h14m43.341269276s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.383215225 (202h14m43.34231248s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.385585359 (202h14m43.344682614s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.388118409 (202h14m43.347215666s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.390509177 (202h14m43.349606432s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Uploading piece... Aug 17 03:40:21.938694025 (202h14m43.89779127s) [upload=210504]: Next returned nil segment Aug 17 03:40:21.938740605 (202h14m43.897837861s) [upload=210504]: Waiting for error group managing segments... Aug 17 03:40:22.087061299 (202h14m44.046158551s) [noise=true , address=45.91.116.6:4997 , piece_id=4EUVUHVUW5HA4WY4F6VI663MPI7QVDO5TK5CNJTDX3PLHOR6IO4A, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.093286222 (202h14m44.052383462s) [noise=true , address=104.36.82.35:4614 , piece_id=QII7R4UHJR2UQXNTOZIUI5ZOAHCGAHH2WPQN4K4C7EKRB22EHG6Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.100022851 (202h14m44.059120106s) [noise=true , address=104.36.82.15:4289 , piece_id=TAODVX3RTLAJGVGH43TDPP55VOQYH73WGOWFXGQIXQSQASAVBYIQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.110175808 (202h14m44.069273065s) [noise=true , address=79.127.219.51:4200 , piece_id=S4JTC3QRSAXUQPPZADFJTHRQG42VUM4UE2CGQ54VKBBAFEADYDEA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.112690417 (202h14m44.071787669s) [noise=true , address=104.36.82.58:4298 , piece_id=UQYYHLOCHX4EYWJZMSUV3B6P2RQ7VPHNLUOGYKJ6S2CDOGSE24SA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.113235068 (202h14m44.072332326s) [noise=true , address=104.36.82.39:4759 , piece_id=7SW6MINWCVIHBVZDEK6EVDKWPD7RPV7EGKSBCI7FNSCBCEGIPNNQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.116860571 (202h14m44.075957818s) [noise=true , address=104.36.82.38:4707 , piece_id=CQISVVVL4KOD4TSYBWRN6E4OOMPSVDNCXE6ZT4DQKLR2TGI5XPMA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.125847240 (202h14m44.084944476s) [noise=true , address=104.36.82.5:4850 , piece_id=4IK4323IBKSD6KICFCDNE76S737WOXOM7O47ZZLC4WJ2JHAPZZQA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.136788037 (202h14m44.095885284s) [noise=true , address=104.36.82.44:4899 , piece_id=T5TEY6BX7JFS34M464PX4V6O4GIKXBUNCFTAA2U2J3Q4YFBWB4DQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.141259899 (202h14m44.100357152s) [noise=true , address=104.36.82.6:4890 , piece_id=3KHSDR6M6CVMMXRDDZW6TWJGBY4ZHKTUMKRTG6D4OF76SVYCTYXQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.157600317 (202h14m44.116697563s) [noise=true , address=77.90.31.18:4837 , piece_id=JCU56BT5FUUNYGA7IUWJT5EEJEA2M5FGO56UYOMGQORXVNF2EFRQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.169517275 (202h14m44.128614514s) [noise=true , address=77.90.31.15:4702 , piece_id=H36PWM3PE2LL4E6MJYBFOWGDUVNHCQZTTCL4CJM6JRG6SJD7SJHA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.170723689 (202h14m44.129820924s) [noise=true , address=77.90.31.11:4561 , piece_id=I5H2EH6WVZQLHNWLMPTNU5SNGEMJKA5ZVBJ4J2GGTTVH5HNA34YA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174287593 (202h14m44.133384838s) [noise=true , address=104.36.82.60:4340 , piece_id=CBS66LU2NULGFPXLPBHWGGY5ZW4SLR57Y3YY7Y2CX4LR6N6DOFBQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: <nil> Aug 17 03:40:22.174338363 (202h14m44.133435608s) [seg_pos={0 0}, upload=210504]: Segment reached optimal threshold of 14 pieces. Aug 17 03:40:22.174565125 (202h14m44.133662363s) [noise=true , address=104.36.82.13:4215 , piece_id=OB5QPNCE73E3MCX45ZJLTJPIOVLKDCCTFXZ6YEUVMPJ6UIX6ZRJQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175117521 (202h14m44.134214757s) [noise=true , address=104.36.82.36:4632 , piece_id=URBDIZPISXZGMTWTWBZGMK3KZJLW2R7WFUTE24M7D7FEVHKIPR4Q, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC): protocol: expected piece hash; context canceled; EOF Aug 17 03:40:22.175174465 (202h14m44.13427171s) [noise=true , address=104.36.82.4:4814 , piece_id=B5WCZH5ZZITTPXDQRFFTUIAHYOB4FHUEBFZGNMR6JTPBEBHOIADA, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ERtrzMuh8rZPsgebtyScmYvNVwz3UKJr5Nu2hLdu6sn8X3qLa): protocol: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52080->104.36.82.4:4814: use of closed network connection Aug 17 03:40:22.175189814 (202h14m44.134287051s) [noise=true , address=193.149.164.143:4696 , piece_id=5TPJU64J3QUDQQPLKPWEQL22JSQHZOBBOQEWRUV376LZXQBSLJZQ, seg_pos={0 0}, upload=210504]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36722->193.149.164.143:4696: use of closed network connection Aug 17 03:40:22.175205651 (202h14m44.134302897s) [seg_pos={0 0}, upload=210504]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 03:40:22.175290900 (202h14m44.134388146s) [upload=210504]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 03:40:22.175296636 (202h14m44.134393881s) [upload=210504]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:40:22.330950759 (202h14m44.290047995s) [upload=210504]: Done uploading segments... Aug 17 03:40:22.330970889 (202h14m44.290068125s) [upload=210504]: Upload finished. err: <nil> Duration: 2.40210015s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................................XX............] duration: 2.638450398s FAIL (2025-08-17 03:40:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.638474425s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 961.992215ms FAIL (2025-08-17 03:39:55 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 02:58:16.481213600 (201h32m38.440310832s) [upload=209799]: Starting upload Aug 17 02:58:16.481294851 (201h32m38.440392083s) [upload=209799]: Done starting upload Aug 17 02:58:16.481388248 (201h32m38.440485481s) [upload=209799]: Uploading segments... Aug 17 02:58:16.481461866 (201h32m38.44055912s) [upload=209799]: Got next segment. Inline: true Aug 17 02:58:16.481467741 (201h32m38.440564988s) [upload=209799]: Waiting for error group managing segments... Aug 17 02:58:16.481531233 (201h32m38.440628465s) [upload=209799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 02:58:16.481537443 (201h32m38.44063469s) [upload=209799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 02:58:16.481541360 (201h32m38.440638606s) [upload=209799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 02:58:16.651015934 (201h32m38.610113181s) [upload=209799]: Done uploading segments... Aug 17 02:58:16.651037402 (201h32m38.610134641s) [upload=209799]: Upload finished. err: <nil> Duration:* 962.010995ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................................XX...........] duration: 2.795821413s FAIL (2025-08-17 03:39:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.795900933s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X.........X...................X........................] duration: 2.422392222s FAIL (2025-08-17 03:39:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:20:04.262329730 (201h54m26.221426987s) [upload=210169]: Starting upload Aug 17 03:20:04.262407161 (201h54m26.221504393s) [upload=210169]: Done starting upload Aug 17 03:20:04.262511453 (201h54m26.221608717s) [upload=210169]: Uploading segments... Aug 17 03:20:04.262773994 (201h54m26.221874962s) [upload=210169]: Got next segment. Inline: false Aug 17 03:20:04.262858819 (201h54m26.221956055s) [upload=210169]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:20:04.262866602 (201h54m26.221963837s) [upload=210169]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:20:04.350065003 (201h54m26.30916226s) [upload=210169]: Next segment err: metaclient: Too Many Requests Aug 17 03:20:04.350093107 (201h54m26.309190341s) [upload=210169]: Done uploading segments... Aug 17 03:20:04.350110583 (201h54m26.309207831s) [upload=210169]: Upload finished. err: metaclient: Too Many Requests Duration: 2.422421915s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 908.273154ms FAIL (2025-08-17 03:39:10 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 02:58:16.481213600 (201h32m38.440310832s) [upload=209799]: Starting upload Aug 17 02:58:16.481294851 (201h32m38.440392083s) [upload=209799]: Done starting upload Aug 17 02:58:16.481388248 (201h32m38.440485481s) [upload=209799]: Uploading segments... Aug 17 02:58:16.481461866 (201h32m38.44055912s) [upload=209799]: Got next segment. Inline: true Aug 17 02:58:16.481467741 (201h32m38.440564988s) [upload=209799]: Waiting for error group managing segments... Aug 17 02:58:16.481531233 (201h32m38.440628465s) [upload=209799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 02:58:16.481537443 (201h32m38.44063469s) [upload=209799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 02:58:16.481541360 (201h32m38.440638606s) [upload=209799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 02:58:16.651015934 (201h32m38.610113181s) [upload=209799]: Done uploading segments... Aug 17 02:58:16.651037402 (201h32m38.610134641s) [upload=209799]: Upload finished. err: <nil> Duration:* 908.392206ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X.........X...................X.......................] duration: 2.479243429s FAIL (2025-08-17 03:39:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:20:04.262329730 (201h54m26.221426987s) [upload=210169]: Starting upload Aug 17 03:20:04.262407161 (201h54m26.221504393s) [upload=210169]: Done starting upload Aug 17 03:20:04.262511453 (201h54m26.221608717s) [upload=210169]: Uploading segments... Aug 17 03:20:04.262773994 (201h54m26.221874962s) [upload=210169]: Got next segment. Inline: false Aug 17 03:20:04.262858819 (201h54m26.221956055s) [upload=210169]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:20:04.262866602 (201h54m26.221963837s) [upload=210169]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:20:04.350065003 (201h54m26.30916226s) [upload=210169]: Next segment err: metaclient: Too Many Requests Aug 17 03:20:04.350093107 (201h54m26.309190341s) [upload=210169]: Done uploading segments... Aug 17 03:20:04.350110583 (201h54m26.309207831s) [upload=210169]: Upload finished. err: metaclient: Too Many Requests Duration: 2.47927724s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................................XX..........] duration: 2.761684531s FAIL (2025-08-17 03:38:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.761714619s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................................XX.........] duration: 2.624862181s FAIL (2025-08-17 03:38:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.62496369s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X.........X...................X......................] duration: 2.388869122s FAIL (2025-08-17 03:38:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:20:04.262329730 (201h54m26.221426987s) [upload=210169]: Starting upload Aug 17 03:20:04.262407161 (201h54m26.221504393s) [upload=210169]: Done starting upload Aug 17 03:20:04.262511453 (201h54m26.221608717s) [upload=210169]: Uploading segments... Aug 17 03:20:04.262773994 (201h54m26.221874962s) [upload=210169]: Got next segment. Inline: false Aug 17 03:20:04.262858819 (201h54m26.221956055s) [upload=210169]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:20:04.262866602 (201h54m26.221963837s) [upload=210169]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:20:04.350065003 (201h54m26.30916226s) [upload=210169]: Next segment err: metaclient: Too Many Requests Aug 17 03:20:04.350093107 (201h54m26.309190341s) [upload=210169]: Done uploading segments... Aug 17 03:20:04.350110583 (201h54m26.309207831s) [upload=210169]: Upload finished. err: metaclient: Too Many Requests Duration: 2.388904035s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 890.440161ms FAIL (2025-08-17 03:37:57 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 02:58:16.481213600 (201h32m38.440310832s) [upload=209799]: Starting upload Aug 17 02:58:16.481294851 (201h32m38.440392083s) [upload=209799]: Done starting upload Aug 17 02:58:16.481388248 (201h32m38.440485481s) [upload=209799]: Uploading segments... Aug 17 02:58:16.481461866 (201h32m38.44055912s) [upload=209799]: Got next segment. Inline: true Aug 17 02:58:16.481467741 (201h32m38.440564988s) [upload=209799]: Waiting for error group managing segments... Aug 17 02:58:16.481531233 (201h32m38.440628465s) [upload=209799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 02:58:16.481537443 (201h32m38.44063469s) [upload=209799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 02:58:16.481541360 (201h32m38.440638606s) [upload=209799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 02:58:16.651015934 (201h32m38.610113181s) [upload=209799]: Done uploading segments... Aug 17 02:58:16.651037402 (201h32m38.610134641s) [upload=209799]: Upload finished. err: <nil> Duration:* 890.522318ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................................XX........] duration: 3.100780806s FAIL (2025-08-17 03:37:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.100803806s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 833.365975ms FAIL (2025-08-17 03:37:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 02:58:16.481213600 (201h32m38.440310832s) [upload=209799]: Starting upload Aug 17 02:58:16.481294851 (201h32m38.440392083s) [upload=209799]: Done starting upload Aug 17 02:58:16.481388248 (201h32m38.440485481s) [upload=209799]: Uploading segments... Aug 17 02:58:16.481461866 (201h32m38.44055912s) [upload=209799]: Got next segment. Inline: true Aug 17 02:58:16.481467741 (201h32m38.440564988s) [upload=209799]: Waiting for error group managing segments... Aug 17 02:58:16.481531233 (201h32m38.440628465s) [upload=209799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 02:58:16.481537443 (201h32m38.44063469s) [upload=209799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 02:58:16.481541360 (201h32m38.440638606s) [upload=209799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 02:58:16.651015934 (201h32m38.610113181s) [upload=209799]: Done uploading segments... Aug 17 02:58:16.651037402 (201h32m38.610134641s) [upload=209799]: Upload finished. err: <nil> Duration:* 833.401344ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [X.......X.........X...................X.....................] duration: 2.336356813s FAIL (2025-08-17 03:37:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:20:04.262329730 (201h54m26.221426987s) [upload=210169]: Starting upload Aug 17 03:20:04.262407161 (201h54m26.221504393s) [upload=210169]: Done starting upload Aug 17 03:20:04.262511453 (201h54m26.221608717s) [upload=210169]: Uploading segments... Aug 17 03:20:04.262773994 (201h54m26.221874962s) [upload=210169]: Got next segment. Inline: false Aug 17 03:20:04.262858819 (201h54m26.221956055s) [upload=210169]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:20:04.262866602 (201h54m26.221963837s) [upload=210169]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:20:04.350065003 (201h54m26.30916226s) [upload=210169]: Next segment err: metaclient: Too Many Requests Aug 17 03:20:04.350093107 (201h54m26.309190341s) [upload=210169]: Done uploading segments... Aug 17 03:20:04.350110583 (201h54m26.309207831s) [upload=210169]: Upload finished. err: metaclient: Too Many Requests Duration: 2.336445553s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 890.352025ms FAIL (2025-08-17 03:36:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 02:58:16.481213600 (201h32m38.440310832s) [upload=209799]: Starting upload Aug 17 02:58:16.481294851 (201h32m38.440392083s) [upload=209799]: Done starting upload Aug 17 02:58:16.481388248 (201h32m38.440485481s) [upload=209799]: Uploading segments... Aug 17 02:58:16.481461866 (201h32m38.44055912s) [upload=209799]: Got next segment. Inline: true Aug 17 02:58:16.481467741 (201h32m38.440564988s) [upload=209799]: Waiting for error group managing segments... Aug 17 02:58:16.481531233 (201h32m38.440628465s) [upload=209799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 02:58:16.481537443 (201h32m38.44063469s) [upload=209799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 02:58:16.481541360 (201h32m38.440638606s) [upload=209799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 02:58:16.651015934 (201h32m38.610113181s) [upload=209799]: Done uploading segments... Aug 17 02:58:16.651037402 (201h32m38.610134641s) [upload=209799]: Upload finished. err: <nil> Duration:* 890.386321ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................XX.......] duration: 2.760742662s FAIL (2025-08-17 03:36:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.760771523s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X.......X.........X...................X....................] duration: 2.623583995s FAIL (2025-08-17 03:36:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:20:04.262329730 (201h54m26.221426987s) [upload=210169]: Starting upload Aug 17 03:20:04.262407161 (201h54m26.221504393s) [upload=210169]: Done starting upload Aug 17 03:20:04.262511453 (201h54m26.221608717s) [upload=210169]: Uploading segments... Aug 17 03:20:04.262773994 (201h54m26.221874962s) [upload=210169]: Got next segment. Inline: false Aug 17 03:20:04.262858819 (201h54m26.221956055s) [upload=210169]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:20:04.262866602 (201h54m26.221963837s) [upload=210169]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 03:20:04.350065003 (201h54m26.30916226s) [upload=210169]: Next segment err: metaclient: Too Many Requests Aug 17 03:20:04.350093107 (201h54m26.309190341s) [upload=210169]: Done uploading segments... Aug 17 03:20:04.350110583 (201h54m26.309207831s) [upload=210169]: Upload finished. err: metaclient: Too Many Requests Duration: 2.623616712s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 872.180546ms FAIL (2025-08-17 03:35:25 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 02:58:16.481213600 (201h32m38.440310832s) [upload=209799]: Starting upload Aug 17 02:58:16.481294851 (201h32m38.440392083s) [upload=209799]: Done starting upload Aug 17 02:58:16.481388248 (201h32m38.440485481s) [upload=209799]: Uploading segments... Aug 17 02:58:16.481461866 (201h32m38.44055912s) [upload=209799]: Got next segment. Inline: true Aug 17 02:58:16.481467741 (201h32m38.440564988s) [upload=209799]: Waiting for error group managing segments... Aug 17 02:58:16.481531233 (201h32m38.440628465s) [upload=209799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 02:58:16.481537443 (201h32m38.44063469s) [upload=209799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 02:58:16.481541360 (201h32m38.440638606s) [upload=209799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 02:58:16.651015934 (201h32m38.610113181s) [upload=209799]: Done uploading segments... Aug 17 02:58:16.651037402 (201h32m38.610134641s) [upload=209799]: Upload finished. err: <nil> Duration:* 872.210714ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)