Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
20 out of 60 is failed (threshold=3) [XX.X.XXXX..X.XXX..X.XX.X.XXXX...X...........................] duration:
3.237054923s
FAIL (2025-07-09 03:17:08 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.237453455s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
12 out of 60 is failed (threshold=3) [..XXX.X.XXX....X..X.X.X.X...................................] duration:
2.935265482s
FAIL (2025-07-09 03:17:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.93529516s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....X.X....................................................] duration:
2.357601802s
FAIL (2025-07-09 03:17:00 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
s) [upload=301814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:36:36.095816711 (377h25m35.979230349s) [upload=301814]: Done uploading segments...
Jul 9 02:36:36.095861438 (377h25m35.979275066s) [upload=301814]: Upload finished. err: <nil>
Jul 9 02:37:33.550348548 (377h26m33.433762165s) [upload=301826]: Starting upload
Jul 9 02:37:33.550443958 (377h26m33.433857587s) [upload=301826]: Done starting upload
Jul 9 02:37:33.550648388 (377h26m33.434062029s) [upload=301826]: Uploading segments...
Jul 9 02:37:33.550844663 (377h26m33.434258301s) [upload=301826]: Got next segment. Inline: false
Jul 9 02:37:33.550917404 (377h26m33.434331039s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:37:33.550924691 (377h26m33.43433833s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:37:33.815909059 (377h26m33.699322692s) [seg_pos={0 0}, upload=301826]: Begin upload segment...
Jul 9 02:37:33.815994368 (377h26m33.699407987s) [seg_pos={0 0}, upload=301826]: Done begin upload segment.
Jul 9 02:37:33.816126929 (377h26m33.699540556s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816186526 (377h26m33.699600158s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816695002 (377h26m33.700108634s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819297193 (377h26m33.702710831s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819764828 (377h26m33.703178443s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.820656075 (377h26m33.704069712s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.822922259 (377h26m33.706335899s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.825191080 (377h26m33.708604717s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.826521734 (377h26m33.709935368s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.827123114 (377h26m33.710536745s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829205661 (377h26m33.712619281s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829794527 (377h26m33.713208161s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.830988926 (377h26m33.714402566s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.831745725 (377h26m33.715159359s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.833607194 (377h26m33.717020827s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.834303078 (377h26m33.717716712s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:34.434801783 (377h26m34.318215418s) [upload=301826]: Next returned nil segment
Jul 9 02:37:34.434824751 (377h26m34.318238389s) [upload=301826]: Waiting for error group managing segments...
Jul 9 02:37:34.571376185 (377h26m34.454789803s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.571556166 (377h26m34.454969779s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.578565129 (377h26m34.461978764s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.588445961 (377h26m34.471859595s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590018122 (377h26m34.473431749s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590534120 (377h26m34.473947751s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.593380073 (377h26m34.476793687s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.595785926 (377h26m34.479199551s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.605401892 (377h26m34.488815504s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.705694279 (377h26m34.589107907s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:38.384942197 (377h26m38.268355813s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:38:03.210659066 (377h27m3.094072698s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210852696 (377h27m3.094266326s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210986553 (377h27m3.094400184s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection
Jul 9 02:38:03.211067271 (377h27m3.094480902s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection
Jul 9 02:38:03.211166349 (377h27m3.094579979s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection
Jul 9 02:38:03.211187224 (377h27m3.094600859s) [seg_pos={0 0}, upload=301826]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul 9 02:38:03.323304657 (377h27m3.206718295s) [upload=301826]: Done uploading segments...
Jul 9 02:38:03.323358440 (377h27m3.206772072s) [upload=301826]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 2.357639395s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
12 out of 60 is failed (threshold=3) [...XXX.X.XXX....X..X.X.X.X..................................] duration:
2.733022567s
FAIL (2025-07-09 03:16:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.733052229s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
20 out of 60 is failed (threshold=3) [.XX.X.XXXX..X.XXX..X.XX.X.XXXX...X..........................] duration:
2.772474114s
FAIL (2025-07-09 03:16:23 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.772507351s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
2.36583085s
FAIL (2025-07-09 03:16:17 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
ting upload
Jul 9 02:48:13.623048621 (377h37m13.506462248s) [upload=301971]: Uploading segments...
Jul 9 02:48:13.623340323 (377h37m13.506753941s) [upload=301971]: Got next segment. Inline: false
Jul 9 02:48:13.623390150 (377h37m13.506803776s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:48:13.623395504 (377h37m13.50680913s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:48:13.817252748 (377h37m13.700666374s) [seg_pos={0 0}, upload=301971]: Begin upload segment...
Jul 9 02:48:13.817328879 (377h37m13.700742516s) [seg_pos={0 0}, upload=301971]: Done begin upload segment.
Jul 9 02:48:13.817359351 (377h37m13.700772986s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817380997 (377h37m13.700794628s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817823520 (377h37m13.701237149s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817891283 (377h37m13.701304922s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818242454 (377h37m13.701656085s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818322245 (377h37m13.701735871s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818545897 (377h37m13.701959539s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818709190 (377h37m13.702122816s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818886009 (377h37m13.702299638s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.819063767 (377h37m13.702477394s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822137924 (377h37m13.705551558s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822570534 (377h37m13.705984166s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822930688 (377h37m13.706344317s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823225875 (377h37m13.706639508s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823562888 (377h37m13.706976522s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823880672 (377h37m13.707294311s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824210482 (377h37m13.707624099s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824584706 (377h37m13.707998336s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:14.400917506 (377h37m14.284331136s) [upload=301971]: Next returned nil segment
Jul 9 02:48:14.400948865 (377h37m14.284362476s) [upload=301971]: Waiting for error group managing segments...
Jul 9 02:48:14.539683847 (377h37m14.423097475s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.545869119 (377h37m14.429282752s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549093850 (377h37m14.432507475s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549682016 (377h37m14.433095629s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559178891 (377h37m14.442592506s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559277031 (377h37m14.442690646s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.561727156 (377h37m14.445140769s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.579411062 (377h37m14.462824682s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.822904625 (377h37m14.706318242s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:15.096350399 (377h37m14.979764016s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:29.510157364 (377h37m29.393570992s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:39.632164275 (377h37m39.515577891s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:44.469621505 (377h37m44.35303512s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:49:13.355633183 (377h38m13.23904681s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:104.36.82.44:4885): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355804399 (377h38m13.239218012s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1hJMMH7ft1vWUYnQeugCHmuYGHd2wnMRm44KyMg997BZajJb3M, address:104.36.82.35:4598): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355849615 (377h38m13.239263241s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:12ssLq5bo7og4z8ZPd28f2H6WUExsWohBBZUnciGaH5KKpcQ25d, address:104.36.82.40:4955): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355894817 (377h38m13.239308443s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:14gJ4XfdRiFLVEixdjSP5QB8F9toGcb6xoxP7kXxX1vXJm5TDg, address:104.36.82.50:4037): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355988218 (377h38m13.239401844s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1JrFvHgEHVLtwK3QYvFTzASCysBCR38qA4Ni8Jvrb1rXUaFy3H, address:104.36.82.38:4696): protocol: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection
Jul 9 02:49:13.356007346 (377h38m13.239420978s) [seg_pos={0 0}, upload=301971]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 02:49:13.464380779 (377h38m13.347794407s) [upload=301971]: Done uploading segments...
Jul 9 02:49:13.464449732 (377h38m13.347863348s) [upload=301971]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.365879975s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [......X.X...................................................] duration:
2.422986531s
FAIL (2025-07-09 03:16:14 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
s) [upload=301814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:36:36.095816711 (377h25m35.979230349s) [upload=301814]: Done uploading segments...
Jul 9 02:36:36.095861438 (377h25m35.979275066s) [upload=301814]: Upload finished. err: <nil>
Jul 9 02:37:33.550348548 (377h26m33.433762165s) [upload=301826]: Starting upload
Jul 9 02:37:33.550443958 (377h26m33.433857587s) [upload=301826]: Done starting upload
Jul 9 02:37:33.550648388 (377h26m33.434062029s) [upload=301826]: Uploading segments...
Jul 9 02:37:33.550844663 (377h26m33.434258301s) [upload=301826]: Got next segment. Inline: false
Jul 9 02:37:33.550917404 (377h26m33.434331039s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:37:33.550924691 (377h26m33.43433833s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:37:33.815909059 (377h26m33.699322692s) [seg_pos={0 0}, upload=301826]: Begin upload segment...
Jul 9 02:37:33.815994368 (377h26m33.699407987s) [seg_pos={0 0}, upload=301826]: Done begin upload segment.
Jul 9 02:37:33.816126929 (377h26m33.699540556s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816186526 (377h26m33.699600158s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816695002 (377h26m33.700108634s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819297193 (377h26m33.702710831s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819764828 (377h26m33.703178443s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.820656075 (377h26m33.704069712s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.822922259 (377h26m33.706335899s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.825191080 (377h26m33.708604717s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.826521734 (377h26m33.709935368s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.827123114 (377h26m33.710536745s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829205661 (377h26m33.712619281s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829794527 (377h26m33.713208161s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.830988926 (377h26m33.714402566s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.831745725 (377h26m33.715159359s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.833607194 (377h26m33.717020827s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.834303078 (377h26m33.717716712s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:34.434801783 (377h26m34.318215418s) [upload=301826]: Next returned nil segment
Jul 9 02:37:34.434824751 (377h26m34.318238389s) [upload=301826]: Waiting for error group managing segments...
Jul 9 02:37:34.571376185 (377h26m34.454789803s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.571556166 (377h26m34.454969779s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.578565129 (377h26m34.461978764s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.588445961 (377h26m34.471859595s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590018122 (377h26m34.473431749s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590534120 (377h26m34.473947751s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.593380073 (377h26m34.476793687s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.595785926 (377h26m34.479199551s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.605401892 (377h26m34.488815504s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.705694279 (377h26m34.589107907s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:38.384942197 (377h26m38.268355813s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:38:03.210659066 (377h27m3.094072698s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210852696 (377h27m3.094266326s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210986553 (377h27m3.094400184s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection
Jul 9 02:38:03.211067271 (377h27m3.094480902s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection
Jul 9 02:38:03.211166349 (377h27m3.094579979s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection
Jul 9 02:38:03.211187224 (377h27m3.094600859s) [seg_pos={0 0}, upload=301826]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul 9 02:38:03.323304657 (377h27m3.206718295s) [upload=301826]: Done uploading segments...
Jul 9 02:38:03.323358440 (377h27m3.206772072s) [upload=301826]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 2.423026869s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
2.458024573s
FAIL (2025-07-09 03:15:58 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
ting upload
Jul 9 02:48:13.623048621 (377h37m13.506462248s) [upload=301971]: Uploading segments...
Jul 9 02:48:13.623340323 (377h37m13.506753941s) [upload=301971]: Got next segment. Inline: false
Jul 9 02:48:13.623390150 (377h37m13.506803776s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:48:13.623395504 (377h37m13.50680913s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:48:13.817252748 (377h37m13.700666374s) [seg_pos={0 0}, upload=301971]: Begin upload segment...
Jul 9 02:48:13.817328879 (377h37m13.700742516s) [seg_pos={0 0}, upload=301971]: Done begin upload segment.
Jul 9 02:48:13.817359351 (377h37m13.700772986s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817380997 (377h37m13.700794628s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817823520 (377h37m13.701237149s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817891283 (377h37m13.701304922s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818242454 (377h37m13.701656085s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818322245 (377h37m13.701735871s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818545897 (377h37m13.701959539s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818709190 (377h37m13.702122816s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818886009 (377h37m13.702299638s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.819063767 (377h37m13.702477394s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822137924 (377h37m13.705551558s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822570534 (377h37m13.705984166s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822930688 (377h37m13.706344317s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823225875 (377h37m13.706639508s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823562888 (377h37m13.706976522s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823880672 (377h37m13.707294311s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824210482 (377h37m13.707624099s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824584706 (377h37m13.707998336s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:14.400917506 (377h37m14.284331136s) [upload=301971]: Next returned nil segment
Jul 9 02:48:14.400948865 (377h37m14.284362476s) [upload=301971]: Waiting for error group managing segments...
Jul 9 02:48:14.539683847 (377h37m14.423097475s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.545869119 (377h37m14.429282752s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549093850 (377h37m14.432507475s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549682016 (377h37m14.433095629s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559178891 (377h37m14.442592506s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559277031 (377h37m14.442690646s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.561727156 (377h37m14.445140769s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.579411062 (377h37m14.462824682s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.822904625 (377h37m14.706318242s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:15.096350399 (377h37m14.979764016s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:29.510157364 (377h37m29.393570992s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:39.632164275 (377h37m39.515577891s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:44.469621505 (377h37m44.35303512s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:49:13.355633183 (377h38m13.23904681s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:104.36.82.44:4885): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355804399 (377h38m13.239218012s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1hJMMH7ft1vWUYnQeugCHmuYGHd2wnMRm44KyMg997BZajJb3M, address:104.36.82.35:4598): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355849615 (377h38m13.239263241s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:12ssLq5bo7og4z8ZPd28f2H6WUExsWohBBZUnciGaH5KKpcQ25d, address:104.36.82.40:4955): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355894817 (377h38m13.239308443s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:14gJ4XfdRiFLVEixdjSP5QB8F9toGcb6xoxP7kXxX1vXJm5TDg, address:104.36.82.50:4037): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355988218 (377h38m13.239401844s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1JrFvHgEHVLtwK3QYvFTzASCysBCR38qA4Ni8Jvrb1rXUaFy3H, address:104.36.82.38:4696): protocol: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection
Jul 9 02:49:13.356007346 (377h38m13.239420978s) [seg_pos={0 0}, upload=301971]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 02:49:13.464380779 (377h38m13.347794407s) [upload=301971]: Done uploading segments...
Jul 9 02:49:13.464449732 (377h38m13.347863348s) [upload=301971]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.458054952s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
12 out of 60 is failed (threshold=3) [....XXX.X.XXX....X..X.X.X.X.................................] duration:
2.852197069s
FAIL (2025-07-09 03:15:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.852816006s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
2.738321282s
FAIL (2025-07-09 03:15:35 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
ting upload
Jul 9 02:48:13.623048621 (377h37m13.506462248s) [upload=301971]: Uploading segments...
Jul 9 02:48:13.623340323 (377h37m13.506753941s) [upload=301971]: Got next segment. Inline: false
Jul 9 02:48:13.623390150 (377h37m13.506803776s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:48:13.623395504 (377h37m13.50680913s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:48:13.817252748 (377h37m13.700666374s) [seg_pos={0 0}, upload=301971]: Begin upload segment...
Jul 9 02:48:13.817328879 (377h37m13.700742516s) [seg_pos={0 0}, upload=301971]: Done begin upload segment.
Jul 9 02:48:13.817359351 (377h37m13.700772986s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817380997 (377h37m13.700794628s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817823520 (377h37m13.701237149s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817891283 (377h37m13.701304922s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818242454 (377h37m13.701656085s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818322245 (377h37m13.701735871s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818545897 (377h37m13.701959539s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818709190 (377h37m13.702122816s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818886009 (377h37m13.702299638s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.819063767 (377h37m13.702477394s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822137924 (377h37m13.705551558s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822570534 (377h37m13.705984166s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822930688 (377h37m13.706344317s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823225875 (377h37m13.706639508s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823562888 (377h37m13.706976522s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823880672 (377h37m13.707294311s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824210482 (377h37m13.707624099s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824584706 (377h37m13.707998336s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:14.400917506 (377h37m14.284331136s) [upload=301971]: Next returned nil segment
Jul 9 02:48:14.400948865 (377h37m14.284362476s) [upload=301971]: Waiting for error group managing segments...
Jul 9 02:48:14.539683847 (377h37m14.423097475s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.545869119 (377h37m14.429282752s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549093850 (377h37m14.432507475s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549682016 (377h37m14.433095629s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559178891 (377h37m14.442592506s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559277031 (377h37m14.442690646s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.561727156 (377h37m14.445140769s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.579411062 (377h37m14.462824682s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.822904625 (377h37m14.706318242s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:15.096350399 (377h37m14.979764016s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:29.510157364 (377h37m29.393570992s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:39.632164275 (377h37m39.515577891s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:44.469621505 (377h37m44.35303512s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:49:13.355633183 (377h38m13.23904681s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:104.36.82.44:4885): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355804399 (377h38m13.239218012s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1hJMMH7ft1vWUYnQeugCHmuYGHd2wnMRm44KyMg997BZajJb3M, address:104.36.82.35:4598): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355849615 (377h38m13.239263241s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:12ssLq5bo7og4z8ZPd28f2H6WUExsWohBBZUnciGaH5KKpcQ25d, address:104.36.82.40:4955): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355894817 (377h38m13.239308443s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:14gJ4XfdRiFLVEixdjSP5QB8F9toGcb6xoxP7kXxX1vXJm5TDg, address:104.36.82.50:4037): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355988218 (377h38m13.239401844s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1JrFvHgEHVLtwK3QYvFTzASCysBCR38qA4Ni8Jvrb1rXUaFy3H, address:104.36.82.38:4696): protocol: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection
Jul 9 02:49:13.356007346 (377h38m13.239420978s) [seg_pos={0 0}, upload=301971]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 02:49:13.464380779 (377h38m13.347794407s) [upload=301971]: Done uploading segments...
Jul 9 02:49:13.464449732 (377h38m13.347863348s) [upload=301971]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.738375267s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......X.X..................................................] duration:
2.360300649s
FAIL (2025-07-09 03:15:33 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
s) [upload=301814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:36:36.095816711 (377h25m35.979230349s) [upload=301814]: Done uploading segments...
Jul 9 02:36:36.095861438 (377h25m35.979275066s) [upload=301814]: Upload finished. err: <nil>
Jul 9 02:37:33.550348548 (377h26m33.433762165s) [upload=301826]: Starting upload
Jul 9 02:37:33.550443958 (377h26m33.433857587s) [upload=301826]: Done starting upload
Jul 9 02:37:33.550648388 (377h26m33.434062029s) [upload=301826]: Uploading segments...
Jul 9 02:37:33.550844663 (377h26m33.434258301s) [upload=301826]: Got next segment. Inline: false
Jul 9 02:37:33.550917404 (377h26m33.434331039s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:37:33.550924691 (377h26m33.43433833s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:37:33.815909059 (377h26m33.699322692s) [seg_pos={0 0}, upload=301826]: Begin upload segment...
Jul 9 02:37:33.815994368 (377h26m33.699407987s) [seg_pos={0 0}, upload=301826]: Done begin upload segment.
Jul 9 02:37:33.816126929 (377h26m33.699540556s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816186526 (377h26m33.699600158s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816695002 (377h26m33.700108634s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819297193 (377h26m33.702710831s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819764828 (377h26m33.703178443s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.820656075 (377h26m33.704069712s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.822922259 (377h26m33.706335899s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.825191080 (377h26m33.708604717s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.826521734 (377h26m33.709935368s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.827123114 (377h26m33.710536745s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829205661 (377h26m33.712619281s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829794527 (377h26m33.713208161s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.830988926 (377h26m33.714402566s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.831745725 (377h26m33.715159359s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.833607194 (377h26m33.717020827s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.834303078 (377h26m33.717716712s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:34.434801783 (377h26m34.318215418s) [upload=301826]: Next returned nil segment
Jul 9 02:37:34.434824751 (377h26m34.318238389s) [upload=301826]: Waiting for error group managing segments...
Jul 9 02:37:34.571376185 (377h26m34.454789803s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.571556166 (377h26m34.454969779s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.578565129 (377h26m34.461978764s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.588445961 (377h26m34.471859595s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590018122 (377h26m34.473431749s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590534120 (377h26m34.473947751s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.593380073 (377h26m34.476793687s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.595785926 (377h26m34.479199551s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.605401892 (377h26m34.488815504s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.705694279 (377h26m34.589107907s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:38.384942197 (377h26m38.268355813s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:38:03.210659066 (377h27m3.094072698s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210852696 (377h27m3.094266326s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210986553 (377h27m3.094400184s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection
Jul 9 02:38:03.211067271 (377h27m3.094480902s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection
Jul 9 02:38:03.211166349 (377h27m3.094579979s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection
Jul 9 02:38:03.211187224 (377h27m3.094600859s) [seg_pos={0 0}, upload=301826]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul 9 02:38:03.323304657 (377h27m3.206718295s) [upload=301826]: Done uploading segments...
Jul 9 02:38:03.323358440 (377h27m3.206772072s) [upload=301826]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 2.360330071s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
21 out of 60 is failed (threshold=3) [X.XX.X.XXXX..X.XXX..X.XX.X.XXXX...X.........................] duration:
2.775785759s
FAIL (2025-07-09 03:15:19 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.77581738s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
22 out of 60 is failed (threshold=3) [XX.XX.X.XXXX..X.XXX..X.XX.X.XXXX...X........................] duration:
3.467910334s
FAIL (2025-07-09 03:14:51 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.467939293s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
22 out of 60 is failed (threshold=3) [.XX.XX.X.XXXX..X.XXX..X.XX.X.XXXX...X.......................] duration:
2.994582754s
FAIL (2025-07-09 03:14:29 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.994650326s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
12 out of 60 is failed (threshold=3) [.....XXX.X.XXX....X..X.X.X.X................................] duration:
2.670345313s
FAIL (2025-07-09 03:14:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.670378442s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
2.395532279s
FAIL (2025-07-09 03:14:25 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
ting upload
Jul 9 02:48:13.623048621 (377h37m13.506462248s) [upload=301971]: Uploading segments...
Jul 9 02:48:13.623340323 (377h37m13.506753941s) [upload=301971]: Got next segment. Inline: false
Jul 9 02:48:13.623390150 (377h37m13.506803776s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:48:13.623395504 (377h37m13.50680913s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:48:13.817252748 (377h37m13.700666374s) [seg_pos={0 0}, upload=301971]: Begin upload segment...
Jul 9 02:48:13.817328879 (377h37m13.700742516s) [seg_pos={0 0}, upload=301971]: Done begin upload segment.
Jul 9 02:48:13.817359351 (377h37m13.700772986s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817380997 (377h37m13.700794628s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817823520 (377h37m13.701237149s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817891283 (377h37m13.701304922s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818242454 (377h37m13.701656085s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818322245 (377h37m13.701735871s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818545897 (377h37m13.701959539s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818709190 (377h37m13.702122816s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818886009 (377h37m13.702299638s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.819063767 (377h37m13.702477394s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822137924 (377h37m13.705551558s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822570534 (377h37m13.705984166s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822930688 (377h37m13.706344317s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823225875 (377h37m13.706639508s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823562888 (377h37m13.706976522s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823880672 (377h37m13.707294311s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824210482 (377h37m13.707624099s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824584706 (377h37m13.707998336s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:14.400917506 (377h37m14.284331136s) [upload=301971]: Next returned nil segment
Jul 9 02:48:14.400948865 (377h37m14.284362476s) [upload=301971]: Waiting for error group managing segments...
Jul 9 02:48:14.539683847 (377h37m14.423097475s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.545869119 (377h37m14.429282752s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549093850 (377h37m14.432507475s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549682016 (377h37m14.433095629s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559178891 (377h37m14.442592506s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559277031 (377h37m14.442690646s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.561727156 (377h37m14.445140769s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.579411062 (377h37m14.462824682s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.822904625 (377h37m14.706318242s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:15.096350399 (377h37m14.979764016s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:29.510157364 (377h37m29.393570992s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:39.632164275 (377h37m39.515577891s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:44.469621505 (377h37m44.35303512s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:49:13.355633183 (377h38m13.23904681s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:104.36.82.44:4885): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355804399 (377h38m13.239218012s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1hJMMH7ft1vWUYnQeugCHmuYGHd2wnMRm44KyMg997BZajJb3M, address:104.36.82.35:4598): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355849615 (377h38m13.239263241s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:12ssLq5bo7og4z8ZPd28f2H6WUExsWohBBZUnciGaH5KKpcQ25d, address:104.36.82.40:4955): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355894817 (377h38m13.239308443s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:14gJ4XfdRiFLVEixdjSP5QB8F9toGcb6xoxP7kXxX1vXJm5TDg, address:104.36.82.50:4037): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355988218 (377h38m13.239401844s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1JrFvHgEHVLtwK3QYvFTzASCysBCR38qA4Ni8Jvrb1rXUaFy3H, address:104.36.82.38:4696): protocol: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection
Jul 9 02:49:13.356007346 (377h38m13.239420978s) [seg_pos={0 0}, upload=301971]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 02:49:13.464380779 (377h38m13.347794407s) [upload=301971]: Done uploading segments...
Jul 9 02:49:13.464449732 (377h38m13.347863348s) [upload=301971]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.395566218s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [........X.X.................................................] duration:
2.383389987s
FAIL (2025-07-09 03:14:24 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
s) [upload=301814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:36:36.095816711 (377h25m35.979230349s) [upload=301814]: Done uploading segments...
Jul 9 02:36:36.095861438 (377h25m35.979275066s) [upload=301814]: Upload finished. err: <nil>
Jul 9 02:37:33.550348548 (377h26m33.433762165s) [upload=301826]: Starting upload
Jul 9 02:37:33.550443958 (377h26m33.433857587s) [upload=301826]: Done starting upload
Jul 9 02:37:33.550648388 (377h26m33.434062029s) [upload=301826]: Uploading segments...
Jul 9 02:37:33.550844663 (377h26m33.434258301s) [upload=301826]: Got next segment. Inline: false
Jul 9 02:37:33.550917404 (377h26m33.434331039s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:37:33.550924691 (377h26m33.43433833s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:37:33.815909059 (377h26m33.699322692s) [seg_pos={0 0}, upload=301826]: Begin upload segment...
Jul 9 02:37:33.815994368 (377h26m33.699407987s) [seg_pos={0 0}, upload=301826]: Done begin upload segment.
Jul 9 02:37:33.816126929 (377h26m33.699540556s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816186526 (377h26m33.699600158s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816695002 (377h26m33.700108634s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819297193 (377h26m33.702710831s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819764828 (377h26m33.703178443s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.820656075 (377h26m33.704069712s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.822922259 (377h26m33.706335899s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.825191080 (377h26m33.708604717s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.826521734 (377h26m33.709935368s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.827123114 (377h26m33.710536745s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829205661 (377h26m33.712619281s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829794527 (377h26m33.713208161s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.830988926 (377h26m33.714402566s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.831745725 (377h26m33.715159359s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.833607194 (377h26m33.717020827s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.834303078 (377h26m33.717716712s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:34.434801783 (377h26m34.318215418s) [upload=301826]: Next returned nil segment
Jul 9 02:37:34.434824751 (377h26m34.318238389s) [upload=301826]: Waiting for error group managing segments...
Jul 9 02:37:34.571376185 (377h26m34.454789803s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.571556166 (377h26m34.454969779s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.578565129 (377h26m34.461978764s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.588445961 (377h26m34.471859595s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590018122 (377h26m34.473431749s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590534120 (377h26m34.473947751s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.593380073 (377h26m34.476793687s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.595785926 (377h26m34.479199551s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.605401892 (377h26m34.488815504s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.705694279 (377h26m34.589107907s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:38.384942197 (377h26m38.268355813s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:38:03.210659066 (377h27m3.094072698s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210852696 (377h27m3.094266326s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210986553 (377h27m3.094400184s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection
Jul 9 02:38:03.211067271 (377h27m3.094480902s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection
Jul 9 02:38:03.211166349 (377h27m3.094579979s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection
Jul 9 02:38:03.211187224 (377h27m3.094600859s) [seg_pos={0 0}, upload=301826]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul 9 02:38:03.323304657 (377h27m3.206718295s) [upload=301826]: Done uploading segments...
Jul 9 02:38:03.323358440 (377h27m3.206772072s) [upload=301826]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 2.383435293s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
4.154478138s
FAIL (2025-07-09 03:14:11 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
ting upload
Jul 9 02:48:13.623048621 (377h37m13.506462248s) [upload=301971]: Uploading segments...
Jul 9 02:48:13.623340323 (377h37m13.506753941s) [upload=301971]: Got next segment. Inline: false
Jul 9 02:48:13.623390150 (377h37m13.506803776s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:48:13.623395504 (377h37m13.50680913s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:48:13.817252748 (377h37m13.700666374s) [seg_pos={0 0}, upload=301971]: Begin upload segment...
Jul 9 02:48:13.817328879 (377h37m13.700742516s) [seg_pos={0 0}, upload=301971]: Done begin upload segment.
Jul 9 02:48:13.817359351 (377h37m13.700772986s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817380997 (377h37m13.700794628s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817823520 (377h37m13.701237149s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817891283 (377h37m13.701304922s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818242454 (377h37m13.701656085s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818322245 (377h37m13.701735871s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818545897 (377h37m13.701959539s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818709190 (377h37m13.702122816s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818886009 (377h37m13.702299638s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.819063767 (377h37m13.702477394s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822137924 (377h37m13.705551558s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822570534 (377h37m13.705984166s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822930688 (377h37m13.706344317s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823225875 (377h37m13.706639508s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823562888 (377h37m13.706976522s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823880672 (377h37m13.707294311s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824210482 (377h37m13.707624099s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824584706 (377h37m13.707998336s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:14.400917506 (377h37m14.284331136s) [upload=301971]: Next returned nil segment
Jul 9 02:48:14.400948865 (377h37m14.284362476s) [upload=301971]: Waiting for error group managing segments...
Jul 9 02:48:14.539683847 (377h37m14.423097475s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.545869119 (377h37m14.429282752s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549093850 (377h37m14.432507475s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549682016 (377h37m14.433095629s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559178891 (377h37m14.442592506s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559277031 (377h37m14.442690646s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.561727156 (377h37m14.445140769s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.579411062 (377h37m14.462824682s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.822904625 (377h37m14.706318242s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:15.096350399 (377h37m14.979764016s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:29.510157364 (377h37m29.393570992s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:39.632164275 (377h37m39.515577891s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:44.469621505 (377h37m44.35303512s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:49:13.355633183 (377h38m13.23904681s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:104.36.82.44:4885): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355804399 (377h38m13.239218012s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1hJMMH7ft1vWUYnQeugCHmuYGHd2wnMRm44KyMg997BZajJb3M, address:104.36.82.35:4598): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355849615 (377h38m13.239263241s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:12ssLq5bo7og4z8ZPd28f2H6WUExsWohBBZUnciGaH5KKpcQ25d, address:104.36.82.40:4955): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355894817 (377h38m13.239308443s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:14gJ4XfdRiFLVEixdjSP5QB8F9toGcb6xoxP7kXxX1vXJm5TDg, address:104.36.82.50:4037): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355988218 (377h38m13.239401844s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1JrFvHgEHVLtwK3QYvFTzASCysBCR38qA4Ni8Jvrb1rXUaFy3H, address:104.36.82.38:4696): protocol: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection
Jul 9 02:49:13.356007346 (377h38m13.239420978s) [seg_pos={0 0}, upload=301971]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 02:49:13.464380779 (377h38m13.347794407s) [upload=301971]: Done uploading segments...
Jul 9 02:49:13.464449732 (377h38m13.347863348s) [upload=301971]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 4.154507433s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
23 out of 60 is failed (threshold=3) [X.XX.XX.X.XXXX..X.XXX..X.XX.X.XXXX...X......................] duration:
2.80892511s
FAIL (2025-07-09 03:13:56 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.808950837s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
2.609984179s
FAIL (2025-07-09 03:13:38 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
ting upload
Jul 9 02:48:13.623048621 (377h37m13.506462248s) [upload=301971]: Uploading segments...
Jul 9 02:48:13.623340323 (377h37m13.506753941s) [upload=301971]: Got next segment. Inline: false
Jul 9 02:48:13.623390150 (377h37m13.506803776s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:48:13.623395504 (377h37m13.50680913s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:48:13.817252748 (377h37m13.700666374s) [seg_pos={0 0}, upload=301971]: Begin upload segment...
Jul 9 02:48:13.817328879 (377h37m13.700742516s) [seg_pos={0 0}, upload=301971]: Done begin upload segment.
Jul 9 02:48:13.817359351 (377h37m13.700772986s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817380997 (377h37m13.700794628s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817823520 (377h37m13.701237149s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817891283 (377h37m13.701304922s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818242454 (377h37m13.701656085s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818322245 (377h37m13.701735871s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818545897 (377h37m13.701959539s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818709190 (377h37m13.702122816s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818886009 (377h37m13.702299638s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.819063767 (377h37m13.702477394s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822137924 (377h37m13.705551558s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822570534 (377h37m13.705984166s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822930688 (377h37m13.706344317s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823225875 (377h37m13.706639508s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823562888 (377h37m13.706976522s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823880672 (377h37m13.707294311s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824210482 (377h37m13.707624099s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824584706 (377h37m13.707998336s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:14.400917506 (377h37m14.284331136s) [upload=301971]: Next returned nil segment
Jul 9 02:48:14.400948865 (377h37m14.284362476s) [upload=301971]: Waiting for error group managing segments...
Jul 9 02:48:14.539683847 (377h37m14.423097475s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.545869119 (377h37m14.429282752s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549093850 (377h37m14.432507475s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549682016 (377h37m14.433095629s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559178891 (377h37m14.442592506s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559277031 (377h37m14.442690646s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.561727156 (377h37m14.445140769s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.579411062 (377h37m14.462824682s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.822904625 (377h37m14.706318242s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:15.096350399 (377h37m14.979764016s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:29.510157364 (377h37m29.393570992s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:39.632164275 (377h37m39.515577891s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:44.469621505 (377h37m44.35303512s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:49:13.355633183 (377h38m13.23904681s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:104.36.82.44:4885): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355804399 (377h38m13.239218012s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1hJMMH7ft1vWUYnQeugCHmuYGHd2wnMRm44KyMg997BZajJb3M, address:104.36.82.35:4598): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355849615 (377h38m13.239263241s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:12ssLq5bo7og4z8ZPd28f2H6WUExsWohBBZUnciGaH5KKpcQ25d, address:104.36.82.40:4955): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355894817 (377h38m13.239308443s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:14gJ4XfdRiFLVEixdjSP5QB8F9toGcb6xoxP7kXxX1vXJm5TDg, address:104.36.82.50:4037): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355988218 (377h38m13.239401844s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1JrFvHgEHVLtwK3QYvFTzASCysBCR38qA4Ni8Jvrb1rXUaFy3H, address:104.36.82.38:4696): protocol: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection
Jul 9 02:49:13.356007346 (377h38m13.239420978s) [seg_pos={0 0}, upload=301971]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 02:49:13.464380779 (377h38m13.347794407s) [upload=301971]: Done uploading segments...
Jul 9 02:49:13.464449732 (377h38m13.347863348s) [upload=301971]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.610029534s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [X.....XXX.X.XXX....X..X.X.X.X...............................] duration:
2.955247459s
FAIL (2025-07-09 03:13:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.955274815s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........X.X................................................] duration:
2.310207786s
FAIL (2025-07-09 03:13:34 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
s) [upload=301814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:36:36.095816711 (377h25m35.979230349s) [upload=301814]: Done uploading segments...
Jul 9 02:36:36.095861438 (377h25m35.979275066s) [upload=301814]: Upload finished. err: <nil>
Jul 9 02:37:33.550348548 (377h26m33.433762165s) [upload=301826]: Starting upload
Jul 9 02:37:33.550443958 (377h26m33.433857587s) [upload=301826]: Done starting upload
Jul 9 02:37:33.550648388 (377h26m33.434062029s) [upload=301826]: Uploading segments...
Jul 9 02:37:33.550844663 (377h26m33.434258301s) [upload=301826]: Got next segment. Inline: false
Jul 9 02:37:33.550917404 (377h26m33.434331039s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:37:33.550924691 (377h26m33.43433833s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:37:33.815909059 (377h26m33.699322692s) [seg_pos={0 0}, upload=301826]: Begin upload segment...
Jul 9 02:37:33.815994368 (377h26m33.699407987s) [seg_pos={0 0}, upload=301826]: Done begin upload segment.
Jul 9 02:37:33.816126929 (377h26m33.699540556s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816186526 (377h26m33.699600158s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816695002 (377h26m33.700108634s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819297193 (377h26m33.702710831s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819764828 (377h26m33.703178443s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.820656075 (377h26m33.704069712s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.822922259 (377h26m33.706335899s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.825191080 (377h26m33.708604717s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.826521734 (377h26m33.709935368s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.827123114 (377h26m33.710536745s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829205661 (377h26m33.712619281s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829794527 (377h26m33.713208161s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.830988926 (377h26m33.714402566s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.831745725 (377h26m33.715159359s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.833607194 (377h26m33.717020827s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.834303078 (377h26m33.717716712s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:34.434801783 (377h26m34.318215418s) [upload=301826]: Next returned nil segment
Jul 9 02:37:34.434824751 (377h26m34.318238389s) [upload=301826]: Waiting for error group managing segments...
Jul 9 02:37:34.571376185 (377h26m34.454789803s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.571556166 (377h26m34.454969779s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.578565129 (377h26m34.461978764s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.588445961 (377h26m34.471859595s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590018122 (377h26m34.473431749s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590534120 (377h26m34.473947751s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.593380073 (377h26m34.476793687s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.595785926 (377h26m34.479199551s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.605401892 (377h26m34.488815504s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.705694279 (377h26m34.589107907s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:38.384942197 (377h26m38.268355813s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:38:03.210659066 (377h27m3.094072698s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210852696 (377h27m3.094266326s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210986553 (377h27m3.094400184s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection
Jul 9 02:38:03.211067271 (377h27m3.094480902s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection
Jul 9 02:38:03.211166349 (377h27m3.094579979s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection
Jul 9 02:38:03.211187224 (377h27m3.094600859s) [seg_pos={0 0}, upload=301826]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul 9 02:38:03.323304657 (377h27m3.206718295s) [upload=301826]: Done uploading segments...
Jul 9 02:38:03.323358440 (377h27m3.206772072s) [upload=301826]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 2.310235601s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
2.341467609s
FAIL (2025-07-09 03:13:14 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
ting upload
Jul 9 02:48:13.623048621 (377h37m13.506462248s) [upload=301971]: Uploading segments...
Jul 9 02:48:13.623340323 (377h37m13.506753941s) [upload=301971]: Got next segment. Inline: false
Jul 9 02:48:13.623390150 (377h37m13.506803776s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:48:13.623395504 (377h37m13.50680913s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:48:13.817252748 (377h37m13.700666374s) [seg_pos={0 0}, upload=301971]: Begin upload segment...
Jul 9 02:48:13.817328879 (377h37m13.700742516s) [seg_pos={0 0}, upload=301971]: Done begin upload segment.
Jul 9 02:48:13.817359351 (377h37m13.700772986s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817380997 (377h37m13.700794628s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817823520 (377h37m13.701237149s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817891283 (377h37m13.701304922s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818242454 (377h37m13.701656085s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818322245 (377h37m13.701735871s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818545897 (377h37m13.701959539s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818709190 (377h37m13.702122816s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818886009 (377h37m13.702299638s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.819063767 (377h37m13.702477394s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822137924 (377h37m13.705551558s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822570534 (377h37m13.705984166s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822930688 (377h37m13.706344317s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823225875 (377h37m13.706639508s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823562888 (377h37m13.706976522s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823880672 (377h37m13.707294311s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824210482 (377h37m13.707624099s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824584706 (377h37m13.707998336s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:14.400917506 (377h37m14.284331136s) [upload=301971]: Next returned nil segment
Jul 9 02:48:14.400948865 (377h37m14.284362476s) [upload=301971]: Waiting for error group managing segments...
Jul 9 02:48:14.539683847 (377h37m14.423097475s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.545869119 (377h37m14.429282752s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549093850 (377h37m14.432507475s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549682016 (377h37m14.433095629s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559178891 (377h37m14.442592506s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559277031 (377h37m14.442690646s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.561727156 (377h37m14.445140769s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.579411062 (377h37m14.462824682s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.822904625 (377h37m14.706318242s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:15.096350399 (377h37m14.979764016s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:29.510157364 (377h37m29.393570992s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:39.632164275 (377h37m39.515577891s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:44.469621505 (377h37m44.35303512s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:49:13.355633183 (377h38m13.23904681s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:104.36.82.44:4885): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355804399 (377h38m13.239218012s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1hJMMH7ft1vWUYnQeugCHmuYGHd2wnMRm44KyMg997BZajJb3M, address:104.36.82.35:4598): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355849615 (377h38m13.239263241s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:12ssLq5bo7og4z8ZPd28f2H6WUExsWohBBZUnciGaH5KKpcQ25d, address:104.36.82.40:4955): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355894817 (377h38m13.239308443s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:14gJ4XfdRiFLVEixdjSP5QB8F9toGcb6xoxP7kXxX1vXJm5TDg, address:104.36.82.50:4037): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355988218 (377h38m13.239401844s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1JrFvHgEHVLtwK3QYvFTzASCysBCR38qA4Ni8Jvrb1rXUaFy3H, address:104.36.82.38:4696): protocol: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection
Jul 9 02:49:13.356007346 (377h38m13.239420978s) [seg_pos={0 0}, upload=301971]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 02:49:13.464380779 (377h38m13.347794407s) [upload=301971]: Done uploading segments...
Jul 9 02:49:13.464449732 (377h38m13.347863348s) [upload=301971]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.341494084s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [.X.....XXX.X.XXX....X..X.X.X.X..............................] duration:
2.792979903s
FAIL (2025-07-09 03:13:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.793070641s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
24 out of 60 is failed (threshold=3) [XX.XX.XX.X.XXXX..X.XXX..X.XX.X.XXXX...X.....................] duration:
3.024907856s
FAIL (2025-07-09 03:12:45 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.024933046s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
25 out of 60 is failed (threshold=3) [XXX.XX.XX.X.XXXX..X.XXX..X.XX.X.XXXX...X....................] duration:
3.262809019s
FAIL (2025-07-09 03:12:28 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.262828424s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........X.X...............................................] duration:
2.278005446s
FAIL (2025-07-09 03:12:22 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
s) [upload=301814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:36:36.095816711 (377h25m35.979230349s) [upload=301814]: Done uploading segments...
Jul 9 02:36:36.095861438 (377h25m35.979275066s) [upload=301814]: Upload finished. err: <nil>
Jul 9 02:37:33.550348548 (377h26m33.433762165s) [upload=301826]: Starting upload
Jul 9 02:37:33.550443958 (377h26m33.433857587s) [upload=301826]: Done starting upload
Jul 9 02:37:33.550648388 (377h26m33.434062029s) [upload=301826]: Uploading segments...
Jul 9 02:37:33.550844663 (377h26m33.434258301s) [upload=301826]: Got next segment. Inline: false
Jul 9 02:37:33.550917404 (377h26m33.434331039s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:37:33.550924691 (377h26m33.43433833s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:37:33.815909059 (377h26m33.699322692s) [seg_pos={0 0}, upload=301826]: Begin upload segment...
Jul 9 02:37:33.815994368 (377h26m33.699407987s) [seg_pos={0 0}, upload=301826]: Done begin upload segment.
Jul 9 02:37:33.816126929 (377h26m33.699540556s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816186526 (377h26m33.699600158s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816695002 (377h26m33.700108634s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819297193 (377h26m33.702710831s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819764828 (377h26m33.703178443s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.820656075 (377h26m33.704069712s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.822922259 (377h26m33.706335899s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.825191080 (377h26m33.708604717s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.826521734 (377h26m33.709935368s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.827123114 (377h26m33.710536745s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829205661 (377h26m33.712619281s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829794527 (377h26m33.713208161s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.830988926 (377h26m33.714402566s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.831745725 (377h26m33.715159359s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.833607194 (377h26m33.717020827s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.834303078 (377h26m33.717716712s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:34.434801783 (377h26m34.318215418s) [upload=301826]: Next returned nil segment
Jul 9 02:37:34.434824751 (377h26m34.318238389s) [upload=301826]: Waiting for error group managing segments...
Jul 9 02:37:34.571376185 (377h26m34.454789803s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.571556166 (377h26m34.454969779s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.578565129 (377h26m34.461978764s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.588445961 (377h26m34.471859595s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590018122 (377h26m34.473431749s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590534120 (377h26m34.473947751s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.593380073 (377h26m34.476793687s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.595785926 (377h26m34.479199551s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.605401892 (377h26m34.488815504s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.705694279 (377h26m34.589107907s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:38.384942197 (377h26m38.268355813s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:38:03.210659066 (377h27m3.094072698s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210852696 (377h27m3.094266326s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210986553 (377h27m3.094400184s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection
Jul 9 02:38:03.211067271 (377h27m3.094480902s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection
Jul 9 02:38:03.211166349 (377h27m3.094579979s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection
Jul 9 02:38:03.211187224 (377h27m3.094600859s) [seg_pos={0 0}, upload=301826]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul 9 02:38:03.323304657 (377h27m3.206718295s) [upload=301826]: Done uploading segments...
Jul 9 02:38:03.323358440 (377h27m3.206772072s) [upload=301826]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 2.278037119s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
2.678395545s
FAIL (2025-07-09 03:12:07 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
ting upload
Jul 9 02:48:13.623048621 (377h37m13.506462248s) [upload=301971]: Uploading segments...
Jul 9 02:48:13.623340323 (377h37m13.506753941s) [upload=301971]: Got next segment. Inline: false
Jul 9 02:48:13.623390150 (377h37m13.506803776s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:48:13.623395504 (377h37m13.50680913s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:48:13.817252748 (377h37m13.700666374s) [seg_pos={0 0}, upload=301971]: Begin upload segment...
Jul 9 02:48:13.817328879 (377h37m13.700742516s) [seg_pos={0 0}, upload=301971]: Done begin upload segment.
Jul 9 02:48:13.817359351 (377h37m13.700772986s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817380997 (377h37m13.700794628s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817823520 (377h37m13.701237149s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817891283 (377h37m13.701304922s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818242454 (377h37m13.701656085s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818322245 (377h37m13.701735871s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818545897 (377h37m13.701959539s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818709190 (377h37m13.702122816s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818886009 (377h37m13.702299638s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.819063767 (377h37m13.702477394s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822137924 (377h37m13.705551558s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822570534 (377h37m13.705984166s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822930688 (377h37m13.706344317s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823225875 (377h37m13.706639508s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823562888 (377h37m13.706976522s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823880672 (377h37m13.707294311s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824210482 (377h37m13.707624099s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824584706 (377h37m13.707998336s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:14.400917506 (377h37m14.284331136s) [upload=301971]: Next returned nil segment
Jul 9 02:48:14.400948865 (377h37m14.284362476s) [upload=301971]: Waiting for error group managing segments...
Jul 9 02:48:14.539683847 (377h37m14.423097475s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.545869119 (377h37m14.429282752s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549093850 (377h37m14.432507475s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549682016 (377h37m14.433095629s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559178891 (377h37m14.442592506s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559277031 (377h37m14.442690646s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.561727156 (377h37m14.445140769s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.579411062 (377h37m14.462824682s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.822904625 (377h37m14.706318242s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:15.096350399 (377h37m14.979764016s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:29.510157364 (377h37m29.393570992s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:39.632164275 (377h37m39.515577891s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:44.469621505 (377h37m44.35303512s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:49:13.355633183 (377h38m13.23904681s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:104.36.82.44:4885): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355804399 (377h38m13.239218012s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1hJMMH7ft1vWUYnQeugCHmuYGHd2wnMRm44KyMg997BZajJb3M, address:104.36.82.35:4598): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355849615 (377h38m13.239263241s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:12ssLq5bo7og4z8ZPd28f2H6WUExsWohBBZUnciGaH5KKpcQ25d, address:104.36.82.40:4955): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355894817 (377h38m13.239308443s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:14gJ4XfdRiFLVEixdjSP5QB8F9toGcb6xoxP7kXxX1vXJm5TDg, address:104.36.82.50:4037): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355988218 (377h38m13.239401844s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1JrFvHgEHVLtwK3QYvFTzASCysBCR38qA4Ni8Jvrb1rXUaFy3H, address:104.36.82.38:4696): protocol: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection
Jul 9 02:49:13.356007346 (377h38m13.239420978s) [seg_pos={0 0}, upload=301971]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 02:49:13.464380779 (377h38m13.347794407s) [upload=301971]: Done uploading segments...
Jul 9 02:49:13.464449732 (377h38m13.347863348s) [upload=301971]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.678494549s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
25 out of 60 is failed (threshold=3) [.XXX.XX.XX.X.XXXX..X.XXX..X.XX.X.XXXX...X...................] duration:
2.833861457s
FAIL (2025-07-09 03:12:01 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.833879325s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [..X.....XXX.X.XXX....X..X.X.X.X.............................] duration:
3.009944591s
FAIL (2025-07-09 03:11:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.009974678s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [...X.....XXX.X.XXX....X..X.X.X.X............................] duration:
2.720888645s
FAIL (2025-07-09 03:11:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.720914084s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
25 out of 60 is failed (threshold=3) [..XXX.XX.XX.X.XXXX..X.XXX..X.XX.X.XXXX...X..................] duration:
2.795188863s
FAIL (2025-07-09 03:11:19 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.795212708s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........X.X..............................................] duration:
2.305856988s
FAIL (2025-07-09 03:11:15 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
s) [upload=301814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:36:36.095816711 (377h25m35.979230349s) [upload=301814]: Done uploading segments...
Jul 9 02:36:36.095861438 (377h25m35.979275066s) [upload=301814]: Upload finished. err: <nil>
Jul 9 02:37:33.550348548 (377h26m33.433762165s) [upload=301826]: Starting upload
Jul 9 02:37:33.550443958 (377h26m33.433857587s) [upload=301826]: Done starting upload
Jul 9 02:37:33.550648388 (377h26m33.434062029s) [upload=301826]: Uploading segments...
Jul 9 02:37:33.550844663 (377h26m33.434258301s) [upload=301826]: Got next segment. Inline: false
Jul 9 02:37:33.550917404 (377h26m33.434331039s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:37:33.550924691 (377h26m33.43433833s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:37:33.815909059 (377h26m33.699322692s) [seg_pos={0 0}, upload=301826]: Begin upload segment...
Jul 9 02:37:33.815994368 (377h26m33.699407987s) [seg_pos={0 0}, upload=301826]: Done begin upload segment.
Jul 9 02:37:33.816126929 (377h26m33.699540556s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816186526 (377h26m33.699600158s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816695002 (377h26m33.700108634s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819297193 (377h26m33.702710831s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819764828 (377h26m33.703178443s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.820656075 (377h26m33.704069712s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.822922259 (377h26m33.706335899s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.825191080 (377h26m33.708604717s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.826521734 (377h26m33.709935368s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.827123114 (377h26m33.710536745s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829205661 (377h26m33.712619281s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829794527 (377h26m33.713208161s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.830988926 (377h26m33.714402566s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.831745725 (377h26m33.715159359s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.833607194 (377h26m33.717020827s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.834303078 (377h26m33.717716712s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:34.434801783 (377h26m34.318215418s) [upload=301826]: Next returned nil segment
Jul 9 02:37:34.434824751 (377h26m34.318238389s) [upload=301826]: Waiting for error group managing segments...
Jul 9 02:37:34.571376185 (377h26m34.454789803s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.571556166 (377h26m34.454969779s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.578565129 (377h26m34.461978764s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.588445961 (377h26m34.471859595s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590018122 (377h26m34.473431749s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590534120 (377h26m34.473947751s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.593380073 (377h26m34.476793687s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.595785926 (377h26m34.479199551s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.605401892 (377h26m34.488815504s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.705694279 (377h26m34.589107907s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:38.384942197 (377h26m38.268355813s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:38:03.210659066 (377h27m3.094072698s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210852696 (377h27m3.094266326s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210986553 (377h27m3.094400184s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection
Jul 9 02:38:03.211067271 (377h27m3.094480902s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection
Jul 9 02:38:03.211166349 (377h27m3.094579979s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection
Jul 9 02:38:03.211187224 (377h27m3.094600859s) [seg_pos={0 0}, upload=301826]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul 9 02:38:03.323304657 (377h27m3.206718295s) [upload=301826]: Done uploading segments...
Jul 9 02:38:03.323358440 (377h27m3.206772072s) [upload=301826]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 2.305963759s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
2.362727262s
FAIL (2025-07-09 03:11:00 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
ting upload
Jul 9 02:48:13.623048621 (377h37m13.506462248s) [upload=301971]: Uploading segments...
Jul 9 02:48:13.623340323 (377h37m13.506753941s) [upload=301971]: Got next segment. Inline: false
Jul 9 02:48:13.623390150 (377h37m13.506803776s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:48:13.623395504 (377h37m13.50680913s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:48:13.817252748 (377h37m13.700666374s) [seg_pos={0 0}, upload=301971]: Begin upload segment...
Jul 9 02:48:13.817328879 (377h37m13.700742516s) [seg_pos={0 0}, upload=301971]: Done begin upload segment.
Jul 9 02:48:13.817359351 (377h37m13.700772986s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817380997 (377h37m13.700794628s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817823520 (377h37m13.701237149s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817891283 (377h37m13.701304922s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818242454 (377h37m13.701656085s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818322245 (377h37m13.701735871s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818545897 (377h37m13.701959539s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818709190 (377h37m13.702122816s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818886009 (377h37m13.702299638s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.819063767 (377h37m13.702477394s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822137924 (377h37m13.705551558s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822570534 (377h37m13.705984166s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822930688 (377h37m13.706344317s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823225875 (377h37m13.706639508s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823562888 (377h37m13.706976522s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823880672 (377h37m13.707294311s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824210482 (377h37m13.707624099s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824584706 (377h37m13.707998336s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:14.400917506 (377h37m14.284331136s) [upload=301971]: Next returned nil segment
Jul 9 02:48:14.400948865 (377h37m14.284362476s) [upload=301971]: Waiting for error group managing segments...
Jul 9 02:48:14.539683847 (377h37m14.423097475s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.545869119 (377h37m14.429282752s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549093850 (377h37m14.432507475s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549682016 (377h37m14.433095629s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559178891 (377h37m14.442592506s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559277031 (377h37m14.442690646s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.561727156 (377h37m14.445140769s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.579411062 (377h37m14.462824682s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.822904625 (377h37m14.706318242s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:15.096350399 (377h37m14.979764016s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:29.510157364 (377h37m29.393570992s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:39.632164275 (377h37m39.515577891s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:44.469621505 (377h37m44.35303512s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:49:13.355633183 (377h38m13.23904681s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:104.36.82.44:4885): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355804399 (377h38m13.239218012s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1hJMMH7ft1vWUYnQeugCHmuYGHd2wnMRm44KyMg997BZajJb3M, address:104.36.82.35:4598): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355849615 (377h38m13.239263241s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:12ssLq5bo7og4z8ZPd28f2H6WUExsWohBBZUnciGaH5KKpcQ25d, address:104.36.82.40:4955): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355894817 (377h38m13.239308443s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:14gJ4XfdRiFLVEixdjSP5QB8F9toGcb6xoxP7kXxX1vXJm5TDg, address:104.36.82.50:4037): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355988218 (377h38m13.239401844s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1JrFvHgEHVLtwK3QYvFTzASCysBCR38qA4Ni8Jvrb1rXUaFy3H, address:104.36.82.38:4696): protocol: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection
Jul 9 02:49:13.356007346 (377h38m13.239420978s) [seg_pos={0 0}, upload=301971]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 02:49:13.464380779 (377h38m13.347794407s) [upload=301971]: Done uploading segments...
Jul 9 02:49:13.464449732 (377h38m13.347863348s) [upload=301971]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.362766519s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [............X.X.............................................] duration:
2.385529206s
FAIL (2025-07-09 03:10:52 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
s) [upload=301814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:36:36.095816711 (377h25m35.979230349s) [upload=301814]: Done uploading segments...
Jul 9 02:36:36.095861438 (377h25m35.979275066s) [upload=301814]: Upload finished. err: <nil>
Jul 9 02:37:33.550348548 (377h26m33.433762165s) [upload=301826]: Starting upload
Jul 9 02:37:33.550443958 (377h26m33.433857587s) [upload=301826]: Done starting upload
Jul 9 02:37:33.550648388 (377h26m33.434062029s) [upload=301826]: Uploading segments...
Jul 9 02:37:33.550844663 (377h26m33.434258301s) [upload=301826]: Got next segment. Inline: false
Jul 9 02:37:33.550917404 (377h26m33.434331039s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:37:33.550924691 (377h26m33.43433833s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:37:33.815909059 (377h26m33.699322692s) [seg_pos={0 0}, upload=301826]: Begin upload segment...
Jul 9 02:37:33.815994368 (377h26m33.699407987s) [seg_pos={0 0}, upload=301826]: Done begin upload segment.
Jul 9 02:37:33.816126929 (377h26m33.699540556s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816186526 (377h26m33.699600158s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816695002 (377h26m33.700108634s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819297193 (377h26m33.702710831s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819764828 (377h26m33.703178443s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.820656075 (377h26m33.704069712s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.822922259 (377h26m33.706335899s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.825191080 (377h26m33.708604717s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.826521734 (377h26m33.709935368s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.827123114 (377h26m33.710536745s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829205661 (377h26m33.712619281s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829794527 (377h26m33.713208161s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.830988926 (377h26m33.714402566s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.831745725 (377h26m33.715159359s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.833607194 (377h26m33.717020827s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.834303078 (377h26m33.717716712s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:34.434801783 (377h26m34.318215418s) [upload=301826]: Next returned nil segment
Jul 9 02:37:34.434824751 (377h26m34.318238389s) [upload=301826]: Waiting for error group managing segments...
Jul 9 02:37:34.571376185 (377h26m34.454789803s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.571556166 (377h26m34.454969779s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.578565129 (377h26m34.461978764s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.588445961 (377h26m34.471859595s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590018122 (377h26m34.473431749s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590534120 (377h26m34.473947751s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.593380073 (377h26m34.476793687s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.595785926 (377h26m34.479199551s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.605401892 (377h26m34.488815504s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.705694279 (377h26m34.589107907s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:38.384942197 (377h26m38.268355813s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:38:03.210659066 (377h27m3.094072698s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210852696 (377h27m3.094266326s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210986553 (377h27m3.094400184s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection
Jul 9 02:38:03.211067271 (377h27m3.094480902s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection
Jul 9 02:38:03.211166349 (377h27m3.094579979s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection
Jul 9 02:38:03.211187224 (377h27m3.094600859s) [seg_pos={0 0}, upload=301826]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul 9 02:38:03.323304657 (377h27m3.206718295s) [upload=301826]: Done uploading segments...
Jul 9 02:38:03.323358440 (377h27m3.206772072s) [upload=301826]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 2.38556005s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
26 out of 60 is failed (threshold=3) [X..XXX.XX.XX.X.XXXX..X.XXX..X.XX.X.XXXX...X.................] duration:
2.896029783s
FAIL (2025-07-09 03:10:48 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.896052587s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [....X.....XXX.X.XXX....X..X.X.X.X...........................] duration:
2.889061101s
FAIL (2025-07-09 03:10:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.889082749s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
2.370724668s
FAIL (2025-07-09 03:10:17 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
ting upload
Jul 9 02:48:13.623048621 (377h37m13.506462248s) [upload=301971]: Uploading segments...
Jul 9 02:48:13.623340323 (377h37m13.506753941s) [upload=301971]: Got next segment. Inline: false
Jul 9 02:48:13.623390150 (377h37m13.506803776s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:48:13.623395504 (377h37m13.50680913s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:48:13.817252748 (377h37m13.700666374s) [seg_pos={0 0}, upload=301971]: Begin upload segment...
Jul 9 02:48:13.817328879 (377h37m13.700742516s) [seg_pos={0 0}, upload=301971]: Done begin upload segment.
Jul 9 02:48:13.817359351 (377h37m13.700772986s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817380997 (377h37m13.700794628s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817823520 (377h37m13.701237149s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817891283 (377h37m13.701304922s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818242454 (377h37m13.701656085s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818322245 (377h37m13.701735871s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818545897 (377h37m13.701959539s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818709190 (377h37m13.702122816s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818886009 (377h37m13.702299638s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.819063767 (377h37m13.702477394s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822137924 (377h37m13.705551558s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822570534 (377h37m13.705984166s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822930688 (377h37m13.706344317s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823225875 (377h37m13.706639508s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823562888 (377h37m13.706976522s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823880672 (377h37m13.707294311s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824210482 (377h37m13.707624099s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824584706 (377h37m13.707998336s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:14.400917506 (377h37m14.284331136s) [upload=301971]: Next returned nil segment
Jul 9 02:48:14.400948865 (377h37m14.284362476s) [upload=301971]: Waiting for error group managing segments...
Jul 9 02:48:14.539683847 (377h37m14.423097475s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.545869119 (377h37m14.429282752s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549093850 (377h37m14.432507475s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549682016 (377h37m14.433095629s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559178891 (377h37m14.442592506s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559277031 (377h37m14.442690646s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.561727156 (377h37m14.445140769s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.579411062 (377h37m14.462824682s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.822904625 (377h37m14.706318242s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:15.096350399 (377h37m14.979764016s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:29.510157364 (377h37m29.393570992s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:39.632164275 (377h37m39.515577891s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:44.469621505 (377h37m44.35303512s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:49:13.355633183 (377h38m13.23904681s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:104.36.82.44:4885): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355804399 (377h38m13.239218012s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1hJMMH7ft1vWUYnQeugCHmuYGHd2wnMRm44KyMg997BZajJb3M, address:104.36.82.35:4598): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355849615 (377h38m13.239263241s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:12ssLq5bo7og4z8ZPd28f2H6WUExsWohBBZUnciGaH5KKpcQ25d, address:104.36.82.40:4955): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355894817 (377h38m13.239308443s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:14gJ4XfdRiFLVEixdjSP5QB8F9toGcb6xoxP7kXxX1vXJm5TDg, address:104.36.82.50:4037): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355988218 (377h38m13.239401844s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1JrFvHgEHVLtwK3QYvFTzASCysBCR38qA4Ni8Jvrb1rXUaFy3H, address:104.36.82.38:4696): protocol: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection
Jul 9 02:49:13.356007346 (377h38m13.239420978s) [seg_pos={0 0}, upload=301971]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 02:49:13.464380779 (377h38m13.347794407s) [upload=301971]: Done uploading segments...
Jul 9 02:49:13.464449732 (377h38m13.347863348s) [upload=301971]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.370794363s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
27 out of 60 is failed (threshold=3) [XX..XXX.XX.XX.X.XXXX..X.XXX..X.XX.X.XXXX...X................] duration:
3.923490091s
FAIL (2025-07-09 03:10:12 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.92352263s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............X.X............................................] duration:
2.288146013s
FAIL (2025-07-09 03:09:50 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
s) [upload=301814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:36:36.095816711 (377h25m35.979230349s) [upload=301814]: Done uploading segments...
Jul 9 02:36:36.095861438 (377h25m35.979275066s) [upload=301814]: Upload finished. err: <nil>
Jul 9 02:37:33.550348548 (377h26m33.433762165s) [upload=301826]: Starting upload
Jul 9 02:37:33.550443958 (377h26m33.433857587s) [upload=301826]: Done starting upload
Jul 9 02:37:33.550648388 (377h26m33.434062029s) [upload=301826]: Uploading segments...
Jul 9 02:37:33.550844663 (377h26m33.434258301s) [upload=301826]: Got next segment. Inline: false
Jul 9 02:37:33.550917404 (377h26m33.434331039s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:37:33.550924691 (377h26m33.43433833s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:37:33.815909059 (377h26m33.699322692s) [seg_pos={0 0}, upload=301826]: Begin upload segment...
Jul 9 02:37:33.815994368 (377h26m33.699407987s) [seg_pos={0 0}, upload=301826]: Done begin upload segment.
Jul 9 02:37:33.816126929 (377h26m33.699540556s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816186526 (377h26m33.699600158s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816695002 (377h26m33.700108634s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819297193 (377h26m33.702710831s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819764828 (377h26m33.703178443s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.820656075 (377h26m33.704069712s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.822922259 (377h26m33.706335899s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.825191080 (377h26m33.708604717s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.826521734 (377h26m33.709935368s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.827123114 (377h26m33.710536745s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829205661 (377h26m33.712619281s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829794527 (377h26m33.713208161s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.830988926 (377h26m33.714402566s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.831745725 (377h26m33.715159359s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.833607194 (377h26m33.717020827s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.834303078 (377h26m33.717716712s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:34.434801783 (377h26m34.318215418s) [upload=301826]: Next returned nil segment
Jul 9 02:37:34.434824751 (377h26m34.318238389s) [upload=301826]: Waiting for error group managing segments...
Jul 9 02:37:34.571376185 (377h26m34.454789803s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.571556166 (377h26m34.454969779s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.578565129 (377h26m34.461978764s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.588445961 (377h26m34.471859595s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590018122 (377h26m34.473431749s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590534120 (377h26m34.473947751s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.593380073 (377h26m34.476793687s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.595785926 (377h26m34.479199551s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.605401892 (377h26m34.488815504s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.705694279 (377h26m34.589107907s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:38.384942197 (377h26m38.268355813s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:38:03.210659066 (377h27m3.094072698s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210852696 (377h27m3.094266326s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210986553 (377h27m3.094400184s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection
Jul 9 02:38:03.211067271 (377h27m3.094480902s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection
Jul 9 02:38:03.211166349 (377h27m3.094579979s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection
Jul 9 02:38:03.211187224 (377h27m3.094600859s) [seg_pos={0 0}, upload=301826]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul 9 02:38:03.323304657 (377h27m3.206718295s) [upload=301826]: Done uploading segments...
Jul 9 02:38:03.323358440 (377h27m3.206772072s) [upload=301826]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 2.288177625s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
2.278165939s
FAIL (2025-07-09 03:09:21 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
ting upload
Jul 9 02:48:13.623048621 (377h37m13.506462248s) [upload=301971]: Uploading segments...
Jul 9 02:48:13.623340323 (377h37m13.506753941s) [upload=301971]: Got next segment. Inline: false
Jul 9 02:48:13.623390150 (377h37m13.506803776s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:48:13.623395504 (377h37m13.50680913s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:48:13.817252748 (377h37m13.700666374s) [seg_pos={0 0}, upload=301971]: Begin upload segment...
Jul 9 02:48:13.817328879 (377h37m13.700742516s) [seg_pos={0 0}, upload=301971]: Done begin upload segment.
Jul 9 02:48:13.817359351 (377h37m13.700772986s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817380997 (377h37m13.700794628s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817823520 (377h37m13.701237149s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817891283 (377h37m13.701304922s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818242454 (377h37m13.701656085s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818322245 (377h37m13.701735871s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818545897 (377h37m13.701959539s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818709190 (377h37m13.702122816s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818886009 (377h37m13.702299638s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.819063767 (377h37m13.702477394s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822137924 (377h37m13.705551558s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822570534 (377h37m13.705984166s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822930688 (377h37m13.706344317s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823225875 (377h37m13.706639508s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823562888 (377h37m13.706976522s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823880672 (377h37m13.707294311s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824210482 (377h37m13.707624099s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824584706 (377h37m13.707998336s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:14.400917506 (377h37m14.284331136s) [upload=301971]: Next returned nil segment
Jul 9 02:48:14.400948865 (377h37m14.284362476s) [upload=301971]: Waiting for error group managing segments...
Jul 9 02:48:14.539683847 (377h37m14.423097475s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.545869119 (377h37m14.429282752s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549093850 (377h37m14.432507475s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549682016 (377h37m14.433095629s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559178891 (377h37m14.442592506s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559277031 (377h37m14.442690646s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.561727156 (377h37m14.445140769s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.579411062 (377h37m14.462824682s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.822904625 (377h37m14.706318242s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:15.096350399 (377h37m14.979764016s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:29.510157364 (377h37m29.393570992s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:39.632164275 (377h37m39.515577891s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:44.469621505 (377h37m44.35303512s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:49:13.355633183 (377h38m13.23904681s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:104.36.82.44:4885): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355804399 (377h38m13.239218012s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1hJMMH7ft1vWUYnQeugCHmuYGHd2wnMRm44KyMg997BZajJb3M, address:104.36.82.35:4598): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355849615 (377h38m13.239263241s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:12ssLq5bo7og4z8ZPd28f2H6WUExsWohBBZUnciGaH5KKpcQ25d, address:104.36.82.40:4955): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355894817 (377h38m13.239308443s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:14gJ4XfdRiFLVEixdjSP5QB8F9toGcb6xoxP7kXxX1vXJm5TDg, address:104.36.82.50:4037): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355988218 (377h38m13.239401844s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1JrFvHgEHVLtwK3QYvFTzASCysBCR38qA4Ni8Jvrb1rXUaFy3H, address:104.36.82.38:4696): protocol: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection
Jul 9 02:49:13.356007346 (377h38m13.239420978s) [seg_pos={0 0}, upload=301971]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 02:49:13.464380779 (377h38m13.347794407s) [upload=301971]: Done uploading segments...
Jul 9 02:49:13.464449732 (377h38m13.347863348s) [upload=301971]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.278195435s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [.....X.....XXX.X.XXX....X..X.X.X.X..........................] duration:
2.719157697s
FAIL (2025-07-09 03:09:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.719183126s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............X.X...........................................] duration:
2.620801018s
FAIL (2025-07-09 03:09:16 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
s) [upload=301814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:36:36.095816711 (377h25m35.979230349s) [upload=301814]: Done uploading segments...
Jul 9 02:36:36.095861438 (377h25m35.979275066s) [upload=301814]: Upload finished. err: <nil>
Jul 9 02:37:33.550348548 (377h26m33.433762165s) [upload=301826]: Starting upload
Jul 9 02:37:33.550443958 (377h26m33.433857587s) [upload=301826]: Done starting upload
Jul 9 02:37:33.550648388 (377h26m33.434062029s) [upload=301826]: Uploading segments...
Jul 9 02:37:33.550844663 (377h26m33.434258301s) [upload=301826]: Got next segment. Inline: false
Jul 9 02:37:33.550917404 (377h26m33.434331039s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:37:33.550924691 (377h26m33.43433833s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:37:33.815909059 (377h26m33.699322692s) [seg_pos={0 0}, upload=301826]: Begin upload segment...
Jul 9 02:37:33.815994368 (377h26m33.699407987s) [seg_pos={0 0}, upload=301826]: Done begin upload segment.
Jul 9 02:37:33.816126929 (377h26m33.699540556s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816186526 (377h26m33.699600158s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816695002 (377h26m33.700108634s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819297193 (377h26m33.702710831s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819764828 (377h26m33.703178443s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.820656075 (377h26m33.704069712s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.822922259 (377h26m33.706335899s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.825191080 (377h26m33.708604717s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.826521734 (377h26m33.709935368s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.827123114 (377h26m33.710536745s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829205661 (377h26m33.712619281s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829794527 (377h26m33.713208161s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.830988926 (377h26m33.714402566s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.831745725 (377h26m33.715159359s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.833607194 (377h26m33.717020827s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.834303078 (377h26m33.717716712s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:34.434801783 (377h26m34.318215418s) [upload=301826]: Next returned nil segment
Jul 9 02:37:34.434824751 (377h26m34.318238389s) [upload=301826]: Waiting for error group managing segments...
Jul 9 02:37:34.571376185 (377h26m34.454789803s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.571556166 (377h26m34.454969779s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.578565129 (377h26m34.461978764s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.588445961 (377h26m34.471859595s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590018122 (377h26m34.473431749s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590534120 (377h26m34.473947751s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.593380073 (377h26m34.476793687s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.595785926 (377h26m34.479199551s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.605401892 (377h26m34.488815504s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.705694279 (377h26m34.589107907s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:38.384942197 (377h26m38.268355813s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:38:03.210659066 (377h27m3.094072698s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210852696 (377h27m3.094266326s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210986553 (377h27m3.094400184s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection
Jul 9 02:38:03.211067271 (377h27m3.094480902s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection
Jul 9 02:38:03.211166349 (377h27m3.094579979s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection
Jul 9 02:38:03.211187224 (377h27m3.094600859s) [seg_pos={0 0}, upload=301826]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul 9 02:38:03.323304657 (377h27m3.206718295s) [upload=301826]: Done uploading segments...
Jul 9 02:38:03.323358440 (377h27m3.206772072s) [upload=301826]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 2.620828786s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [......X.....XXX.X.XXX....X..X.X.X.X.........................] duration:
3.046530443s
FAIL (2025-07-09 03:09:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.046559938s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
28 out of 60 is failed (threshold=3) [XXX..XXX.XX.XX.X.XXXX..X.XXX..X.XX.X.XXXX...X...............] duration:
3.423434708s
FAIL (2025-07-09 03:09:04 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.423468917s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............X.X..........................................] duration:
2.26243109s
FAIL (2025-07-09 03:08:56 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
s) [upload=301814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:36:36.095816711 (377h25m35.979230349s) [upload=301814]: Done uploading segments...
Jul 9 02:36:36.095861438 (377h25m35.979275066s) [upload=301814]: Upload finished. err: <nil>
Jul 9 02:37:33.550348548 (377h26m33.433762165s) [upload=301826]: Starting upload
Jul 9 02:37:33.550443958 (377h26m33.433857587s) [upload=301826]: Done starting upload
Jul 9 02:37:33.550648388 (377h26m33.434062029s) [upload=301826]: Uploading segments...
Jul 9 02:37:33.550844663 (377h26m33.434258301s) [upload=301826]: Got next segment. Inline: false
Jul 9 02:37:33.550917404 (377h26m33.434331039s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:37:33.550924691 (377h26m33.43433833s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:37:33.815909059 (377h26m33.699322692s) [seg_pos={0 0}, upload=301826]: Begin upload segment...
Jul 9 02:37:33.815994368 (377h26m33.699407987s) [seg_pos={0 0}, upload=301826]: Done begin upload segment.
Jul 9 02:37:33.816126929 (377h26m33.699540556s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816186526 (377h26m33.699600158s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816695002 (377h26m33.700108634s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819297193 (377h26m33.702710831s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819764828 (377h26m33.703178443s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.820656075 (377h26m33.704069712s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.822922259 (377h26m33.706335899s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.825191080 (377h26m33.708604717s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.826521734 (377h26m33.709935368s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.827123114 (377h26m33.710536745s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829205661 (377h26m33.712619281s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829794527 (377h26m33.713208161s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.830988926 (377h26m33.714402566s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.831745725 (377h26m33.715159359s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.833607194 (377h26m33.717020827s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.834303078 (377h26m33.717716712s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:34.434801783 (377h26m34.318215418s) [upload=301826]: Next returned nil segment
Jul 9 02:37:34.434824751 (377h26m34.318238389s) [upload=301826]: Waiting for error group managing segments...
Jul 9 02:37:34.571376185 (377h26m34.454789803s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.571556166 (377h26m34.454969779s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.578565129 (377h26m34.461978764s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.588445961 (377h26m34.471859595s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590018122 (377h26m34.473431749s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590534120 (377h26m34.473947751s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.593380073 (377h26m34.476793687s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.595785926 (377h26m34.479199551s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.605401892 (377h26m34.488815504s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.705694279 (377h26m34.589107907s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:38.384942197 (377h26m38.268355813s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:38:03.210659066 (377h27m3.094072698s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210852696 (377h27m3.094266326s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210986553 (377h27m3.094400184s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection
Jul 9 02:38:03.211067271 (377h27m3.094480902s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection
Jul 9 02:38:03.211166349 (377h27m3.094579979s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection
Jul 9 02:38:03.211187224 (377h27m3.094600859s) [seg_pos={0 0}, upload=301826]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul 9 02:38:03.323304657 (377h27m3.206718295s) [upload=301826]: Done uploading segments...
Jul 9 02:38:03.323358440 (377h27m3.206772072s) [upload=301826]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 2.262459905s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
2.494713553s
FAIL (2025-07-09 03:08:52 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
ting upload
Jul 9 02:48:13.623048621 (377h37m13.506462248s) [upload=301971]: Uploading segments...
Jul 9 02:48:13.623340323 (377h37m13.506753941s) [upload=301971]: Got next segment. Inline: false
Jul 9 02:48:13.623390150 (377h37m13.506803776s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:48:13.623395504 (377h37m13.50680913s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:48:13.817252748 (377h37m13.700666374s) [seg_pos={0 0}, upload=301971]: Begin upload segment...
Jul 9 02:48:13.817328879 (377h37m13.700742516s) [seg_pos={0 0}, upload=301971]: Done begin upload segment.
Jul 9 02:48:13.817359351 (377h37m13.700772986s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817380997 (377h37m13.700794628s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817823520 (377h37m13.701237149s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817891283 (377h37m13.701304922s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818242454 (377h37m13.701656085s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818322245 (377h37m13.701735871s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818545897 (377h37m13.701959539s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818709190 (377h37m13.702122816s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818886009 (377h37m13.702299638s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.819063767 (377h37m13.702477394s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822137924 (377h37m13.705551558s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822570534 (377h37m13.705984166s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822930688 (377h37m13.706344317s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823225875 (377h37m13.706639508s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823562888 (377h37m13.706976522s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823880672 (377h37m13.707294311s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824210482 (377h37m13.707624099s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824584706 (377h37m13.707998336s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:14.400917506 (377h37m14.284331136s) [upload=301971]: Next returned nil segment
Jul 9 02:48:14.400948865 (377h37m14.284362476s) [upload=301971]: Waiting for error group managing segments...
Jul 9 02:48:14.539683847 (377h37m14.423097475s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.545869119 (377h37m14.429282752s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549093850 (377h37m14.432507475s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549682016 (377h37m14.433095629s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559178891 (377h37m14.442592506s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559277031 (377h37m14.442690646s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.561727156 (377h37m14.445140769s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.579411062 (377h37m14.462824682s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.822904625 (377h37m14.706318242s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:15.096350399 (377h37m14.979764016s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:29.510157364 (377h37m29.393570992s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:39.632164275 (377h37m39.515577891s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:44.469621505 (377h37m44.35303512s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:49:13.355633183 (377h38m13.23904681s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:104.36.82.44:4885): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355804399 (377h38m13.239218012s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1hJMMH7ft1vWUYnQeugCHmuYGHd2wnMRm44KyMg997BZajJb3M, address:104.36.82.35:4598): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355849615 (377h38m13.239263241s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:12ssLq5bo7og4z8ZPd28f2H6WUExsWohBBZUnciGaH5KKpcQ25d, address:104.36.82.40:4955): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355894817 (377h38m13.239308443s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:14gJ4XfdRiFLVEixdjSP5QB8F9toGcb6xoxP7kXxX1vXJm5TDg, address:104.36.82.50:4037): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355988218 (377h38m13.239401844s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1JrFvHgEHVLtwK3QYvFTzASCysBCR38qA4Ni8Jvrb1rXUaFy3H, address:104.36.82.38:4696): protocol: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection
Jul 9 02:49:13.356007346 (377h38m13.239420978s) [seg_pos={0 0}, upload=301971]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 02:49:13.464380779 (377h38m13.347794407s) [upload=301971]: Done uploading segments...
Jul 9 02:49:13.464449732 (377h38m13.347863348s) [upload=301971]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.49474327s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
2.335483912s
FAIL (2025-07-09 03:08:33 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
ting upload
Jul 9 02:48:13.623048621 (377h37m13.506462248s) [upload=301971]: Uploading segments...
Jul 9 02:48:13.623340323 (377h37m13.506753941s) [upload=301971]: Got next segment. Inline: false
Jul 9 02:48:13.623390150 (377h37m13.506803776s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:48:13.623395504 (377h37m13.50680913s) [upload=301971]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:48:13.817252748 (377h37m13.700666374s) [seg_pos={0 0}, upload=301971]: Begin upload segment...
Jul 9 02:48:13.817328879 (377h37m13.700742516s) [seg_pos={0 0}, upload=301971]: Done begin upload segment.
Jul 9 02:48:13.817359351 (377h37m13.700772986s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817380997 (377h37m13.700794628s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817823520 (377h37m13.701237149s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.817891283 (377h37m13.701304922s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818242454 (377h37m13.701656085s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818322245 (377h37m13.701735871s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818545897 (377h37m13.701959539s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818709190 (377h37m13.702122816s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.818886009 (377h37m13.702299638s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.819063767 (377h37m13.702477394s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822137924 (377h37m13.705551558s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822570534 (377h37m13.705984166s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.822930688 (377h37m13.706344317s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823225875 (377h37m13.706639508s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823562888 (377h37m13.706976522s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.823880672 (377h37m13.707294311s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824210482 (377h37m13.707624099s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:13.824584706 (377h37m13.707998336s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Uploading piece...
Jul 9 02:48:14.400917506 (377h37m14.284331136s) [upload=301971]: Next returned nil segment
Jul 9 02:48:14.400948865 (377h37m14.284362476s) [upload=301971]: Waiting for error group managing segments...
Jul 9 02:48:14.539683847 (377h37m14.423097475s) [noise=true , address=77.90.31.14:4679 , piece_id=B5WKU3LFJC4RX5BBGISCGHGDVY2FCRUYK3LHSJPFEYIF7LSAT5NA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.545869119 (377h37m14.429282752s) [noise=true , address=77.90.31.16:4758 , piece_id=E74XD4JFGC4MQXHZ4HWJ35RTVXSZL7XKNG7D2YSGHW3FD6NRC4DQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549093850 (377h37m14.432507475s) [noise=true , address=77.90.31.11:4556 , piece_id=PEV6YNDVYJ6F5ZXBLUGB5K5MXPZ4B6YISCKM44Z7DWQ3L5GLTR2Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.549682016 (377h37m14.433095629s) [noise=true , address=77.90.31.15:4731 , piece_id=SBKWBYURU7VZDMU3OP36HXOOA2SCXHEHUUUVJXFTQFDM6FGD3QYA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559178891 (377h37m14.442592506s) [noise=true , address=77.90.31.19:4852 , piece_id=VA3E5FIKHLWK6O3I5PPDJS52MKB77JC733CVQ46WFLAGIJDVGDZA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.559277031 (377h37m14.442690646s) [noise=true , address=77.90.31.18:4832 , piece_id=DKEOHPYKXWZLJHPFZCERN7KVRY5AQYGZEV5YVW24PPHFKRGY7RAQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.561727156 (377h37m14.445140769s) [noise=true , address=77.90.31.10:4528 , piece_id=Z45WKOXUE65MKNUTAFEBOUUDNCVWPRGI7D6SPOZX7F6DVZXLQI5A, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.579411062 (377h37m14.462824682s) [noise=true , address=77.90.31.13:4653 , piece_id=7K7IYONUOVWCSQMCOZMJ4OENYB3A24WXDQJQ3YFD6X7KZRTHEQMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:14.822904625 (377h37m14.706318242s) [noise=true , address=156.146.43.225:4449 , piece_id=H4TXOQLWVMT6CLQMMFKXP2QLPDIPVUVE2TRMGIXJX3JZWHQJUN3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:15.096350399 (377h37m14.979764016s) [noise=true , address=79.127.205.235:4662 , piece_id=IBJFBZYO3LSKLWJLZROD2CIB7YQXTTOVYRVUKXXODKFH7AT2CEGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:29.510157364 (377h37m29.393570992s) [noise=true , address=104.36.82.2:4763 , piece_id=ITBKWOMUINXGYF3QSKL3EA646UIBTGBZ5K25MZAWC63WYY4HVNEQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:39.632164275 (377h37m39.515577891s) [noise=true , address=104.36.82.12:4161 , piece_id=GAC4GFKQXQIPTEI2VPGAABDI2AYV7GZSXN4L7LY75KW2R6BXKRCQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:48:44.469621505 (377h37m44.35303512s) [noise=true , address=104.36.82.19:4431 , piece_id=XR2KRXZS7VEII53QLWH3NP6RHZ3CL653JPNUWST4EWPOS2ZP2W3Q, seg_pos={0 0}, upload=301971]: Done uploading piece. err: <nil>
Jul 9 02:49:13.355633183 (377h38m13.23904681s) [noise=true , address=104.36.82.44:4885 , piece_id=QWKTSS4453IJRCPI6BCY4OYUMEYQDCHI7HP45HWHPHJOQN5L4INA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:104.36.82.44:4885): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355804399 (377h38m13.239218012s) [noise=true , address=104.36.82.35:4598 , piece_id=UW7DOPZFL6TDXN4DQGL6OMEWGVF76SWE5QYB7TQFS4DL7NPAUGGA, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1hJMMH7ft1vWUYnQeugCHmuYGHd2wnMRm44KyMg997BZajJb3M, address:104.36.82.35:4598): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355849615 (377h38m13.239263241s) [noise=true , address=104.36.82.40:4955 , piece_id=3AO226MG7GQ4VJGTFCQKJV7LADBDR52DHIBDWDKACYQQOQDJLWMQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:12ssLq5bo7og4z8ZPd28f2H6WUExsWohBBZUnciGaH5KKpcQ25d, address:104.36.82.40:4955): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355894817 (377h38m13.239308443s) [noise=true , address=104.36.82.50:4037 , piece_id=JSQH5SHZVBZJPVB7W2CPQCX7LOU7GKXM3CBKE7N52BXVXBEF3JTQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:14gJ4XfdRiFLVEixdjSP5QB8F9toGcb6xoxP7kXxX1vXJm5TDg, address:104.36.82.50:4037): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:49:13.355988218 (377h38m13.239401844s) [noise=true , address=104.36.82.38:4696 , piece_id=QSUPCFVH2E2AR2AY5AO6X4KBNPTUTYDNM4U53PEGJYWDAPH4BPBQ, seg_pos={0 0}, upload=301971]: Done uploading piece. err: ecclient: upload failed (node:1JrFvHgEHVLtwK3QYvFTzASCysBCR38qA4Ni8Jvrb1rXUaFy3H, address:104.36.82.38:4696): protocol: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47232->104.36.82.38:4696: use of closed network connection
Jul 9 02:49:13.356007346 (377h38m13.239420978s) [seg_pos={0 0}, upload=301971]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 02:49:13.464380779 (377h38m13.347794407s) [upload=301971]: Done uploading segments...
Jul 9 02:49:13.464449732 (377h38m13.347863348s) [upload=301971]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.335514977s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
29 out of 60 is failed (threshold=3) [XXXX..XXX.XX.XX.X.XXXX..X.XXX..X.XX.X.XXXX...X..............] duration:
3.221120057s
FAIL (2025-07-09 03:08:24 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.221215861s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [................X.X.........................................] duration:
2.29563664s
FAIL (2025-07-09 03:08:20 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
s) [upload=301814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:36:36.095816711 (377h25m35.979230349s) [upload=301814]: Done uploading segments...
Jul 9 02:36:36.095861438 (377h25m35.979275066s) [upload=301814]: Upload finished. err: <nil>
Jul 9 02:37:33.550348548 (377h26m33.433762165s) [upload=301826]: Starting upload
Jul 9 02:37:33.550443958 (377h26m33.433857587s) [upload=301826]: Done starting upload
Jul 9 02:37:33.550648388 (377h26m33.434062029s) [upload=301826]: Uploading segments...
Jul 9 02:37:33.550844663 (377h26m33.434258301s) [upload=301826]: Got next segment. Inline: false
Jul 9 02:37:33.550917404 (377h26m33.434331039s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:37:33.550924691 (377h26m33.43433833s) [upload=301826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:37:33.815909059 (377h26m33.699322692s) [seg_pos={0 0}, upload=301826]: Begin upload segment...
Jul 9 02:37:33.815994368 (377h26m33.699407987s) [seg_pos={0 0}, upload=301826]: Done begin upload segment.
Jul 9 02:37:33.816126929 (377h26m33.699540556s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816186526 (377h26m33.699600158s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.816695002 (377h26m33.700108634s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819297193 (377h26m33.702710831s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.819764828 (377h26m33.703178443s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.820656075 (377h26m33.704069712s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.822922259 (377h26m33.706335899s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.825191080 (377h26m33.708604717s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.826521734 (377h26m33.709935368s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.827123114 (377h26m33.710536745s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829205661 (377h26m33.712619281s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.829794527 (377h26m33.713208161s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.830988926 (377h26m33.714402566s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.831745725 (377h26m33.715159359s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.833607194 (377h26m33.717020827s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:33.834303078 (377h26m33.717716712s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Uploading piece...
Jul 9 02:37:34.434801783 (377h26m34.318215418s) [upload=301826]: Next returned nil segment
Jul 9 02:37:34.434824751 (377h26m34.318238389s) [upload=301826]: Waiting for error group managing segments...
Jul 9 02:37:34.571376185 (377h26m34.454789803s) [noise=true , address=45.91.116.13:4153 , piece_id=Z6XVAGF5J4X7SS6JZ3B5SQEZC5WIGELGIZPGBMFTO6GXDCWQCDQQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.571556166 (377h26m34.454969779s) [noise=true , address=104.36.82.9:4006 , piece_id=VYX4ZOBKJDOBPDDU6XXBLMY7CBYFXHUCFCQ6HSVFOJM674P6ZWTQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.578565129 (377h26m34.461978764s) [noise=true , address=45.91.116.6:4974 , piece_id=F452K44DF3QDGFZS4BCZIFE5L3K6MLGHID242BT2CVK5FIAH5L6A, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.588445961 (377h26m34.471859595s) [noise=true , address=45.91.116.10:4083 , piece_id=RPNK2JI6GXPAYAJ56AXMQSFHW3WNCMYRAKWOLJM7PZOOLFEL34HA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590018122 (377h26m34.473431749s) [noise=true , address=77.90.31.13:4626 , piece_id=IK2IS6NTA74ZALIYY2SOCB2KBCMUXYLMYXCPFJFUYLKCNDSLBHFA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.590534120 (377h26m34.473947751s) [noise=true , address=45.91.116.7:4009 , piece_id=I4IBQHFN6RUJOQP6253J6VUSVXZEXXOCXSMB4E5XZDTNE4QB26KA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.593380073 (377h26m34.476793687s) [noise=true , address=77.90.31.19:4841 , piece_id=P2URXIABU2ZJ2RDXPTYWUDHDVKGF7MLA2J32CTAPGB4YDU3457ZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.595785926 (377h26m34.479199551s) [noise=true , address=77.90.31.21:4879 , piece_id=YMM726IO7X3OPX6YOY3X3F6XYB6L3PSMIFO67OVJVRGPCYBK2LVA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.605401892 (377h26m34.488815504s) [noise=true , address=77.90.31.19:4842 , piece_id=YC2ON4JKTBTHVO42CPCQMFHF6JTVSDV4RVZEXL7NFNIFPUZEIPXQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:34.705694279 (377h26m34.589107907s) [noise=true , address=45.91.116.4:4938 , piece_id=FGYBHC7J2MUWXR6RQ7ZKM4ZZFU75T6SQWCXUPPDX4PL7JSDDAGQA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:37:38.384942197 (377h26m38.268355813s) [noise=true , address=45.91.116.7:4010 , piece_id=QJYX72LLDYA6MGLTN5O6HGKU74KTDSFI3V2MBCBWGOWJQSK7SWKA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: <nil>
Jul 9 02:38:03.210659066 (377h27m3.094072698s) [noise=true , address=104.36.82.5:4838 , piece_id=HZHRW7OZTVDXA2T4Q3SSJAAKW4MFQOSDPNZ6XTH7LR3HGZWWSCZA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210852696 (377h27m3.094266326s) [noise=true , address=104.36.82.8:4970 , piece_id=TNZJWTR4TMUHC26ADZVZJNPWRVWCP4MIICA53FG7KV7YJHOZ4QSA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 02:38:03.210986553 (377h27m3.094400184s) [noise=true , address=104.36.82.2:4730 , piece_id=3WOQE32LZN3Q7EVPBW5SXP33U2APCP5OFCNIBMS7V7CIBFVWU4IQ, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47666->104.36.82.2:4730: use of closed network connection
Jul 9 02:38:03.211067271 (377h27m3.094480902s) [noise=true , address=104.36.82.4:4803 , piece_id=4UZ44IPO2IKLLB2T3EGUIWKFP6C67QTZRA5D2KP6RNHFO2P2BGHA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48930->104.36.82.4:4803: use of closed network connection
Jul 9 02:38:03.211166349 (377h27m3.094579979s) [noise=true , address=104.36.82.4:4801 , piece_id=LIWPQRP7QIHNDGI2VYMGOBV5LHSFDNBZLH6T3YV34VGQCLA2AMEA, seg_pos={0 0}, upload=301826]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41238->104.36.82.4:4801: use of closed network connection
Jul 9 02:38:03.211187224 (377h27m3.094600859s) [seg_pos={0 0}, upload=301826]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul 9 02:38:03.323304657 (377h27m3.206718295s) [upload=301826]: Done uploading segments...
Jul 9 02:38:03.323358440 (377h27m3.206772072s) [upload=301826]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 2.29572785s
core
uplink
us-select-1
|