Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
635.603275ms
FAIL (2025-06-24 15:23:55 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 635.631994ms
edge
authservice
http
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
3.085422334s
FAIL (2025-06-24 15:23:54 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 3.085942564s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
908.913771ms
FAIL (2025-06-24 15:23:50 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 909.16815ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
2.842729039s
FAIL (2025-06-24 15:23:38 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 2.842748996s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
2.299149451s
FAIL (2025-06-24 15:23:25 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments...
Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil>
Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload
Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload
Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments...
Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false
Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment...
Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment.
Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment
Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments...
Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces.
Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled
Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments...
Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil>
Duration: 2.299177722s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.................................X....................X.....] duration:
2.686103282s
FAIL (2025-06-24 15:23:12 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
======LOGS======
(30h9m57.569988099s) [upload=24239]: Got next segment. Inline: false
Jun 24 15:20:57.686619305 (30h9m57.570032934s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 15:20:57.686638888 (30h9m57.570052517s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 15:20:57.928224467 (30h9m57.811638102s) [seg_pos={0 0}, upload=24239]: Begin upload segment...
Jun 24 15:20:57.928301679 (30h9m57.811715319s) [seg_pos={0 0}, upload=24239]: Done begin upload segment.
Jun 24 15:20:57.928337777 (30h9m57.811751414s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928406629 (30h9m57.811820265s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928863827 (30h9m57.812277463s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929025477 (30h9m57.81243913s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929286116 (30h9m57.812699754s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929564621 (30h9m57.812978257s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929730217 (30h9m57.813143832s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930153445 (30h9m57.813567083s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930242518 (30h9m57.813656153s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930630185 (30h9m57.81404382s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930638645 (30h9m57.814052284s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931017196 (30h9m57.814430834s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931132509 (30h9m57.814546156s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931366452 (30h9m57.81478014s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931564988 (30h9m57.814978626s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931785078 (30h9m57.815198715s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932199806 (30h9m57.815613443s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932597121 (30h9m57.816010761s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:58.481271254 (30h9m58.364684873s) [upload=24239]: Next returned nil segment
Jun 24 15:20:58.481353797 (30h9m58.364767428s) [upload=24239]: Waiting for error group managing segments...
Jun 24 15:20:58.655262482 (30h9m58.53867612s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.657655175 (30h9m58.541068803s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.662068056 (30h9m58.545481674s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.666971693 (30h9m58.550385332s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.673154612 (30h9m58.556568247s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.677822189 (30h9m58.561235808s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.684887890 (30h9m58.568301504s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.690528248 (30h9m58.573941867s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.701981027 (30h9m58.58539464s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.709947868 (30h9m58.593361508s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.713191677 (30h9m58.596605305s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.729487718 (30h9m58.612901353s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.749570660 (30h9m58.632984293s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846275930 (30h9m58.729689563s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846340737 (30h9m58.729754371s) [seg_pos={0 0}, upload=24239]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:20:58.846632560 (30h9m58.730046195s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jWewg226E4BrBxqcK38n4ZjHx3toQeNdbc5HeU2EoykztXwyX): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846836814 (30h9m58.730250441s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126nRFki7uLyG9xwY8t6BeAumn8ME2HwccdC2EAwsiEV1p2juZ9): protocol: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection
Jun 24 15:20:58.846884087 (30h9m58.730297723s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eTYJjTHAyM7gvmxrtzHMjFxCCQRzs6fgkVX7Rjj1ze9nVXjwr): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846899396 (30h9m58.730313025s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H9d6LpjXV9CHf3BenRN8M1uYNMF6ULNFnZzprJZsktsXCdoQb): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846918006 (30h9m58.730331619s) [seg_pos={0 0}, upload=24239]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:20:58.847033031 (30h9m58.73044668s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:20:58.847051227 (30h9m58.730464859s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:21:38.513456613 (30h10m38.396870249s) [upload=24239]: Done uploading segments...
Jun 24 15:21:38.513493889 (30h10m38.396907528s) [upload=24239]: Upload finished. err: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
Duration: 2.686138569s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
651.427276ms
FAIL (2025-06-24 15:23:10 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 651.454739ms
edge
authservice
http
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
877.021416ms
FAIL (2025-06-24 15:23:02 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 877.071275ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
3.712443731s
FAIL (2025-06-24 15:22:59 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 3.712477896s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..................................X....................X....] duration:
2.580427755s
FAIL (2025-06-24 15:22:54 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
======LOGS======
(30h9m57.569988099s) [upload=24239]: Got next segment. Inline: false
Jun 24 15:20:57.686619305 (30h9m57.570032934s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 15:20:57.686638888 (30h9m57.570052517s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 15:20:57.928224467 (30h9m57.811638102s) [seg_pos={0 0}, upload=24239]: Begin upload segment...
Jun 24 15:20:57.928301679 (30h9m57.811715319s) [seg_pos={0 0}, upload=24239]: Done begin upload segment.
Jun 24 15:20:57.928337777 (30h9m57.811751414s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928406629 (30h9m57.811820265s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928863827 (30h9m57.812277463s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929025477 (30h9m57.81243913s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929286116 (30h9m57.812699754s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929564621 (30h9m57.812978257s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929730217 (30h9m57.813143832s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930153445 (30h9m57.813567083s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930242518 (30h9m57.813656153s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930630185 (30h9m57.81404382s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930638645 (30h9m57.814052284s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931017196 (30h9m57.814430834s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931132509 (30h9m57.814546156s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931366452 (30h9m57.81478014s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931564988 (30h9m57.814978626s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931785078 (30h9m57.815198715s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932199806 (30h9m57.815613443s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932597121 (30h9m57.816010761s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:58.481271254 (30h9m58.364684873s) [upload=24239]: Next returned nil segment
Jun 24 15:20:58.481353797 (30h9m58.364767428s) [upload=24239]: Waiting for error group managing segments...
Jun 24 15:20:58.655262482 (30h9m58.53867612s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.657655175 (30h9m58.541068803s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.662068056 (30h9m58.545481674s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.666971693 (30h9m58.550385332s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.673154612 (30h9m58.556568247s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.677822189 (30h9m58.561235808s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.684887890 (30h9m58.568301504s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.690528248 (30h9m58.573941867s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.701981027 (30h9m58.58539464s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.709947868 (30h9m58.593361508s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.713191677 (30h9m58.596605305s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.729487718 (30h9m58.612901353s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.749570660 (30h9m58.632984293s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846275930 (30h9m58.729689563s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846340737 (30h9m58.729754371s) [seg_pos={0 0}, upload=24239]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:20:58.846632560 (30h9m58.730046195s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jWewg226E4BrBxqcK38n4ZjHx3toQeNdbc5HeU2EoykztXwyX): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846836814 (30h9m58.730250441s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126nRFki7uLyG9xwY8t6BeAumn8ME2HwccdC2EAwsiEV1p2juZ9): protocol: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection
Jun 24 15:20:58.846884087 (30h9m58.730297723s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eTYJjTHAyM7gvmxrtzHMjFxCCQRzs6fgkVX7Rjj1ze9nVXjwr): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846899396 (30h9m58.730313025s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H9d6LpjXV9CHf3BenRN8M1uYNMF6ULNFnZzprJZsktsXCdoQb): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846918006 (30h9m58.730331619s) [seg_pos={0 0}, upload=24239]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:20:58.847033031 (30h9m58.73044668s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:20:58.847051227 (30h9m58.730464859s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:21:38.513456613 (30h10m38.396870249s) [upload=24239]: Done uploading segments...
Jun 24 15:21:38.513493889 (30h10m38.396907528s) [upload=24239]: Upload finished. err: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
Duration: 2.580471734s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
2.395121827s
FAIL (2025-06-24 15:22:48 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments...
Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil>
Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload
Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload
Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments...
Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false
Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment...
Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment.
Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment
Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments...
Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces.
Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled
Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments...
Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil>
Duration: 2.39523909s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...................................X....................X...] duration:
2.618277018s
FAIL (2025-06-24 15:22:39 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
======LOGS======
(30h9m57.569988099s) [upload=24239]: Got next segment. Inline: false
Jun 24 15:20:57.686619305 (30h9m57.570032934s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 15:20:57.686638888 (30h9m57.570052517s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 15:20:57.928224467 (30h9m57.811638102s) [seg_pos={0 0}, upload=24239]: Begin upload segment...
Jun 24 15:20:57.928301679 (30h9m57.811715319s) [seg_pos={0 0}, upload=24239]: Done begin upload segment.
Jun 24 15:20:57.928337777 (30h9m57.811751414s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928406629 (30h9m57.811820265s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928863827 (30h9m57.812277463s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929025477 (30h9m57.81243913s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929286116 (30h9m57.812699754s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929564621 (30h9m57.812978257s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929730217 (30h9m57.813143832s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930153445 (30h9m57.813567083s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930242518 (30h9m57.813656153s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930630185 (30h9m57.81404382s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930638645 (30h9m57.814052284s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931017196 (30h9m57.814430834s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931132509 (30h9m57.814546156s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931366452 (30h9m57.81478014s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931564988 (30h9m57.814978626s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931785078 (30h9m57.815198715s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932199806 (30h9m57.815613443s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932597121 (30h9m57.816010761s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:58.481271254 (30h9m58.364684873s) [upload=24239]: Next returned nil segment
Jun 24 15:20:58.481353797 (30h9m58.364767428s) [upload=24239]: Waiting for error group managing segments...
Jun 24 15:20:58.655262482 (30h9m58.53867612s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.657655175 (30h9m58.541068803s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.662068056 (30h9m58.545481674s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.666971693 (30h9m58.550385332s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.673154612 (30h9m58.556568247s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.677822189 (30h9m58.561235808s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.684887890 (30h9m58.568301504s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.690528248 (30h9m58.573941867s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.701981027 (30h9m58.58539464s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.709947868 (30h9m58.593361508s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.713191677 (30h9m58.596605305s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.729487718 (30h9m58.612901353s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.749570660 (30h9m58.632984293s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846275930 (30h9m58.729689563s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846340737 (30h9m58.729754371s) [seg_pos={0 0}, upload=24239]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:20:58.846632560 (30h9m58.730046195s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jWewg226E4BrBxqcK38n4ZjHx3toQeNdbc5HeU2EoykztXwyX): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846836814 (30h9m58.730250441s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126nRFki7uLyG9xwY8t6BeAumn8ME2HwccdC2EAwsiEV1p2juZ9): protocol: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection
Jun 24 15:20:58.846884087 (30h9m58.730297723s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eTYJjTHAyM7gvmxrtzHMjFxCCQRzs6fgkVX7Rjj1ze9nVXjwr): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846899396 (30h9m58.730313025s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H9d6LpjXV9CHf3BenRN8M1uYNMF6ULNFnZzprJZsktsXCdoQb): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846918006 (30h9m58.730331619s) [seg_pos={0 0}, upload=24239]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:20:58.847033031 (30h9m58.73044668s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:20:58.847051227 (30h9m58.730464859s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:21:38.513456613 (30h10m38.396870249s) [upload=24239]: Done uploading segments...
Jun 24 15:21:38.513493889 (30h10m38.396907528s) [upload=24239]: Upload finished. err: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
Duration: 2.618331047s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
828.225847ms
FAIL (2025-06-24 15:22:28 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 828.259686ms
core
uplink
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
1.323145588s
FAIL (2025-06-24 15:22:12 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.323180991s
edge
authservice
http
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
3.207412493s
FAIL (2025-06-24 15:22:09 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 3.207514422s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [....................................X....................X..] duration:
2.405951116s
FAIL (2025-06-24 15:22:05 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
======LOGS======
(30h9m57.569988099s) [upload=24239]: Got next segment. Inline: false
Jun 24 15:20:57.686619305 (30h9m57.570032934s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 15:20:57.686638888 (30h9m57.570052517s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 15:20:57.928224467 (30h9m57.811638102s) [seg_pos={0 0}, upload=24239]: Begin upload segment...
Jun 24 15:20:57.928301679 (30h9m57.811715319s) [seg_pos={0 0}, upload=24239]: Done begin upload segment.
Jun 24 15:20:57.928337777 (30h9m57.811751414s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928406629 (30h9m57.811820265s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928863827 (30h9m57.812277463s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929025477 (30h9m57.81243913s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929286116 (30h9m57.812699754s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929564621 (30h9m57.812978257s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929730217 (30h9m57.813143832s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930153445 (30h9m57.813567083s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930242518 (30h9m57.813656153s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930630185 (30h9m57.81404382s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930638645 (30h9m57.814052284s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931017196 (30h9m57.814430834s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931132509 (30h9m57.814546156s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931366452 (30h9m57.81478014s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931564988 (30h9m57.814978626s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931785078 (30h9m57.815198715s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932199806 (30h9m57.815613443s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932597121 (30h9m57.816010761s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:58.481271254 (30h9m58.364684873s) [upload=24239]: Next returned nil segment
Jun 24 15:20:58.481353797 (30h9m58.364767428s) [upload=24239]: Waiting for error group managing segments...
Jun 24 15:20:58.655262482 (30h9m58.53867612s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.657655175 (30h9m58.541068803s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.662068056 (30h9m58.545481674s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.666971693 (30h9m58.550385332s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.673154612 (30h9m58.556568247s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.677822189 (30h9m58.561235808s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.684887890 (30h9m58.568301504s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.690528248 (30h9m58.573941867s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.701981027 (30h9m58.58539464s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.709947868 (30h9m58.593361508s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.713191677 (30h9m58.596605305s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.729487718 (30h9m58.612901353s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.749570660 (30h9m58.632984293s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846275930 (30h9m58.729689563s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846340737 (30h9m58.729754371s) [seg_pos={0 0}, upload=24239]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:20:58.846632560 (30h9m58.730046195s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jWewg226E4BrBxqcK38n4ZjHx3toQeNdbc5HeU2EoykztXwyX): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846836814 (30h9m58.730250441s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126nRFki7uLyG9xwY8t6BeAumn8ME2HwccdC2EAwsiEV1p2juZ9): protocol: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection
Jun 24 15:20:58.846884087 (30h9m58.730297723s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eTYJjTHAyM7gvmxrtzHMjFxCCQRzs6fgkVX7Rjj1ze9nVXjwr): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846899396 (30h9m58.730313025s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H9d6LpjXV9CHf3BenRN8M1uYNMF6ULNFnZzprJZsktsXCdoQb): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846918006 (30h9m58.730331619s) [seg_pos={0 0}, upload=24239]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:20:58.847033031 (30h9m58.73044668s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:20:58.847051227 (30h9m58.730464859s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:21:38.513456613 (30h10m38.396870249s) [upload=24239]: Done uploading segments...
Jun 24 15:21:38.513493889 (30h10m38.396907528s) [upload=24239]: Upload finished. err: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
Duration: 2.406027417s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
2.383203686s
FAIL (2025-06-24 15:21:48 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments...
Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil>
Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload
Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload
Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments...
Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false
Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment...
Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment.
Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment
Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments...
Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces.
Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled
Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments...
Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil>
Duration: 2.383238676s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
880.474059ms
FAIL (2025-06-24 15:21:45 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 880.519329ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.....................................X....................X.] duration:
2.91972652s
FAIL (2025-06-24 15:21:41 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
======LOGS======
(30h9m57.569988099s) [upload=24239]: Got next segment. Inline: false
Jun 24 15:20:57.686619305 (30h9m57.570032934s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 15:20:57.686638888 (30h9m57.570052517s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 15:20:57.928224467 (30h9m57.811638102s) [seg_pos={0 0}, upload=24239]: Begin upload segment...
Jun 24 15:20:57.928301679 (30h9m57.811715319s) [seg_pos={0 0}, upload=24239]: Done begin upload segment.
Jun 24 15:20:57.928337777 (30h9m57.811751414s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928406629 (30h9m57.811820265s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928863827 (30h9m57.812277463s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929025477 (30h9m57.81243913s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929286116 (30h9m57.812699754s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929564621 (30h9m57.812978257s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929730217 (30h9m57.813143832s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930153445 (30h9m57.813567083s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930242518 (30h9m57.813656153s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930630185 (30h9m57.81404382s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930638645 (30h9m57.814052284s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931017196 (30h9m57.814430834s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931132509 (30h9m57.814546156s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931366452 (30h9m57.81478014s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931564988 (30h9m57.814978626s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931785078 (30h9m57.815198715s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932199806 (30h9m57.815613443s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932597121 (30h9m57.816010761s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:58.481271254 (30h9m58.364684873s) [upload=24239]: Next returned nil segment
Jun 24 15:20:58.481353797 (30h9m58.364767428s) [upload=24239]: Waiting for error group managing segments...
Jun 24 15:20:58.655262482 (30h9m58.53867612s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.657655175 (30h9m58.541068803s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.662068056 (30h9m58.545481674s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.666971693 (30h9m58.550385332s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.673154612 (30h9m58.556568247s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.677822189 (30h9m58.561235808s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.684887890 (30h9m58.568301504s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.690528248 (30h9m58.573941867s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.701981027 (30h9m58.58539464s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.709947868 (30h9m58.593361508s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.713191677 (30h9m58.596605305s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.729487718 (30h9m58.612901353s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.749570660 (30h9m58.632984293s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846275930 (30h9m58.729689563s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846340737 (30h9m58.729754371s) [seg_pos={0 0}, upload=24239]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:20:58.846632560 (30h9m58.730046195s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jWewg226E4BrBxqcK38n4ZjHx3toQeNdbc5HeU2EoykztXwyX): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846836814 (30h9m58.730250441s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126nRFki7uLyG9xwY8t6BeAumn8ME2HwccdC2EAwsiEV1p2juZ9): protocol: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection
Jun 24 15:20:58.846884087 (30h9m58.730297723s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eTYJjTHAyM7gvmxrtzHMjFxCCQRzs6fgkVX7Rjj1ze9nVXjwr): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846899396 (30h9m58.730313025s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H9d6LpjXV9CHf3BenRN8M1uYNMF6ULNFnZzprJZsktsXCdoQb): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846918006 (30h9m58.730331619s) [seg_pos={0 0}, upload=24239]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:20:58.847033031 (30h9m58.73044668s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:20:58.847051227 (30h9m58.730464859s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:21:38.513456613 (30h10m38.396870249s) [upload=24239]: Done uploading segments...
Jun 24 15:21:38.513493889 (30h10m38.396907528s) [upload=24239]: Upload finished. err: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
Duration: 2.919758355s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [......................................X....................X] duration:
41.104165367s
FAIL (2025-06-24 15:21:38 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
======LOGS======
(30h9m57.569988099s) [upload=24239]: Got next segment. Inline: false
Jun 24 15:20:57.686619305 (30h9m57.570032934s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 15:20:57.686638888 (30h9m57.570052517s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 15:20:57.928224467 (30h9m57.811638102s) [seg_pos={0 0}, upload=24239]: Begin upload segment...
Jun 24 15:20:57.928301679 (30h9m57.811715319s) [seg_pos={0 0}, upload=24239]: Done begin upload segment.
Jun 24 15:20:57.928337777 (30h9m57.811751414s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928406629 (30h9m57.811820265s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928863827 (30h9m57.812277463s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929025477 (30h9m57.81243913s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929286116 (30h9m57.812699754s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929564621 (30h9m57.812978257s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929730217 (30h9m57.813143832s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930153445 (30h9m57.813567083s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930242518 (30h9m57.813656153s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930630185 (30h9m57.81404382s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930638645 (30h9m57.814052284s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931017196 (30h9m57.814430834s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931132509 (30h9m57.814546156s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931366452 (30h9m57.81478014s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931564988 (30h9m57.814978626s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931785078 (30h9m57.815198715s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932199806 (30h9m57.815613443s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932597121 (30h9m57.816010761s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:58.481271254 (30h9m58.364684873s) [upload=24239]: Next returned nil segment
Jun 24 15:20:58.481353797 (30h9m58.364767428s) [upload=24239]: Waiting for error group managing segments...
Jun 24 15:20:58.655262482 (30h9m58.53867612s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.657655175 (30h9m58.541068803s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.662068056 (30h9m58.545481674s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.666971693 (30h9m58.550385332s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.673154612 (30h9m58.556568247s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.677822189 (30h9m58.561235808s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.684887890 (30h9m58.568301504s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.690528248 (30h9m58.573941867s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.701981027 (30h9m58.58539464s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.709947868 (30h9m58.593361508s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.713191677 (30h9m58.596605305s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.729487718 (30h9m58.612901353s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.749570660 (30h9m58.632984293s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846275930 (30h9m58.729689563s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846340737 (30h9m58.729754371s) [seg_pos={0 0}, upload=24239]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:20:58.846632560 (30h9m58.730046195s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jWewg226E4BrBxqcK38n4ZjHx3toQeNdbc5HeU2EoykztXwyX): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846836814 (30h9m58.730250441s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126nRFki7uLyG9xwY8t6BeAumn8ME2HwccdC2EAwsiEV1p2juZ9): protocol: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection
Jun 24 15:20:58.846884087 (30h9m58.730297723s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eTYJjTHAyM7gvmxrtzHMjFxCCQRzs6fgkVX7Rjj1ze9nVXjwr): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846899396 (30h9m58.730313025s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H9d6LpjXV9CHf3BenRN8M1uYNMF6ULNFnZzprJZsktsXCdoQb): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846918006 (30h9m58.730331619s) [seg_pos={0 0}, upload=24239]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:20:58.847033031 (30h9m58.73044668s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:20:58.847051227 (30h9m58.730464859s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:21:38.513456613 (30h10m38.396870249s) [upload=24239]: Done uploading segments...
Jun 24 15:21:38.513493889 (30h10m38.396907528s) [upload=24239]: Upload finished. err: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
Duration: 41.104267405s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
3.145009216s
FAIL (2025-06-24 15:21:31 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 3.145031688s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
2.255980522s
FAIL (2025-06-24 15:21:19 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments...
Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil>
Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload
Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload
Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments...
Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false
Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment...
Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment.
Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment
Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments...
Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces.
Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled
Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments...
Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil>
Duration: 2.256016341s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
852.701143ms
FAIL (2025-06-24 15:21:05 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 852.733692ms
core
uplink
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
717.604335ms
FAIL (2025-06-24 15:20:58 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 717.643911ms
edge
authservice
http
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
2.2951461s
FAIL (2025-06-24 15:20:56 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments...
Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil>
Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload
Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload
Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments...
Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false
Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment...
Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment.
Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment
Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments...
Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces.
Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled
Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments...
Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil>
Duration: 2.29517668s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
32.324960937s
FAIL (2025-06-24 15:20:54 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 32.324997344s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
2.477921628s
FAIL (2025-06-24 15:20:18 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
38.710079463 (29h51m38.593493084s) [seg_pos={0 0}, upload=23996]: Done begin upload segment.
Jun 24 15:02:38.710129041 (29h51m38.59354267s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.710138687 (29h51m38.593552328s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.710690489 (29h51m38.594104106s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.710713779 (29h51m38.594127417s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.711106997 (29h51m38.594520623s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.711531376 (29h51m38.594945005s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712074545 (29h51m38.59548818s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712237061 (29h51m38.595650701s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712469485 (29h51m38.595883111s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712638648 (29h51m38.596052285s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712933855 (29h51m38.596347485s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.713378835 (29h51m38.596792472s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.713813822 (29h51m38.597227453s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.713901007 (29h51m38.597314653s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.714349767 (29h51m38.597763401s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.714694647 (29h51m38.59810828s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.715041886 (29h51m38.598455519s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.715462120 (29h51m38.59887574s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:39.251183574 (29h51m39.134597192s) [upload=23996]: Next returned nil segment
Jun 24 15:02:39.251207849 (29h51m39.134621467s) [upload=23996]: Waiting for error group managing segments...
Jun 24 15:02:39.412884907 (29h51m39.296298523s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.417725225 (29h51m39.301138871s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.419058833 (29h51m39.302472446s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.428264821 (29h51m39.311678434s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.430916704 (29h51m39.314330317s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.441395180 (29h51m39.324808793s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.443411058 (29h51m39.326824675s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.453557685 (29h51m39.336971305s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.461322189 (29h51m39.344735804s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.473469757 (29h51m39.356883373s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.473537251 (29h51m39.356950869s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.486281273 (29h51m39.369694894s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.571776596 (29h51m39.455190214s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.653833016 (29h51m39.537246631s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.653863972 (29h51m39.537277598s) [seg_pos={0 0}, upload=23996]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:02:39.654154687 (29h51m39.537568301s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: expected piece hash; context canceled; EOF
Jun 24 15:02:39.654204520 (29h51m39.537618156s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ac3N4hMJ66bTx2dpHBDT6W5SEGnPNfwyonwmvAANR8mM8vvpY): protocol: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection
Jun 24 15:02:39.654312246 (29h51m39.537725915s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection
Jun 24 15:02:39.654354065 (29h51m39.537767695s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZgWv9RA5EpAHfxzBWLpTj8un93LPU8syqxM9gBQQomMP9MRwG): protocol: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection
Jun 24 15:02:39.654401800 (29h51m39.537815437s) [seg_pos={0 0}, upload=23996]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:02:39.654504756 (29h51m39.537918371s) [upload=23996]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:02:39.654510115 (29h51m39.537923741s) [upload=23996]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:02:39.810746072 (29h51m39.694159689s) [upload=23996]: Done uploading segments...
Jun 24 15:02:39.810767325 (29h51m39.694180941s) [upload=23996]: Upload finished. err: <nil>
Duration: 2.47797425s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
918.12676ms
FAIL (2025-06-24 15:20:14 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 918.158609ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
2.323788088s
FAIL (2025-06-24 15:19:57 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments...
Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil>
Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload
Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload
Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments...
Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false
Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment...
Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment.
Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment
Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments...
Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces.
Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled
Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments...
Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil>
Duration: 2.323822079s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
2.410760306s
FAIL (2025-06-24 15:19:57 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
38.710079463 (29h51m38.593493084s) [seg_pos={0 0}, upload=23996]: Done begin upload segment.
Jun 24 15:02:38.710129041 (29h51m38.59354267s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.710138687 (29h51m38.593552328s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.710690489 (29h51m38.594104106s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.710713779 (29h51m38.594127417s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.711106997 (29h51m38.594520623s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.711531376 (29h51m38.594945005s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712074545 (29h51m38.59548818s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712237061 (29h51m38.595650701s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712469485 (29h51m38.595883111s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712638648 (29h51m38.596052285s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712933855 (29h51m38.596347485s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.713378835 (29h51m38.596792472s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.713813822 (29h51m38.597227453s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.713901007 (29h51m38.597314653s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.714349767 (29h51m38.597763401s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.714694647 (29h51m38.59810828s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.715041886 (29h51m38.598455519s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.715462120 (29h51m38.59887574s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:39.251183574 (29h51m39.134597192s) [upload=23996]: Next returned nil segment
Jun 24 15:02:39.251207849 (29h51m39.134621467s) [upload=23996]: Waiting for error group managing segments...
Jun 24 15:02:39.412884907 (29h51m39.296298523s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.417725225 (29h51m39.301138871s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.419058833 (29h51m39.302472446s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.428264821 (29h51m39.311678434s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.430916704 (29h51m39.314330317s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.441395180 (29h51m39.324808793s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.443411058 (29h51m39.326824675s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.453557685 (29h51m39.336971305s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.461322189 (29h51m39.344735804s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.473469757 (29h51m39.356883373s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.473537251 (29h51m39.356950869s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.486281273 (29h51m39.369694894s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.571776596 (29h51m39.455190214s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.653833016 (29h51m39.537246631s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.653863972 (29h51m39.537277598s) [seg_pos={0 0}, upload=23996]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:02:39.654154687 (29h51m39.537568301s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: expected piece hash; context canceled; EOF
Jun 24 15:02:39.654204520 (29h51m39.537618156s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ac3N4hMJ66bTx2dpHBDT6W5SEGnPNfwyonwmvAANR8mM8vvpY): protocol: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection
Jun 24 15:02:39.654312246 (29h51m39.537725915s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection
Jun 24 15:02:39.654354065 (29h51m39.537767695s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZgWv9RA5EpAHfxzBWLpTj8un93LPU8syqxM9gBQQomMP9MRwG): protocol: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection
Jun 24 15:02:39.654401800 (29h51m39.537815437s) [seg_pos={0 0}, upload=23996]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:02:39.654504756 (29h51m39.537918371s) [upload=23996]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:02:39.654510115 (29h51m39.537923741s) [upload=23996]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:02:39.810746072 (29h51m39.694159689s) [upload=23996]: Done uploading segments...
Jun 24 15:02:39.810767325 (29h51m39.694180941s) [upload=23996]: Upload finished. err: <nil>
Duration: 2.410786149s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
867.517132ms
FAIL (2025-06-24 15:19:46 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 867.546404ms
edge
authservice
http
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
1.014975745s
FAIL (2025-06-24 15:19:40 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 1.015024364s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
3.184588394s
FAIL (2025-06-24 15:19:26 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 3.184619493s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
938.021804ms
FAIL (2025-06-24 15:19:20 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 938.055786ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
2.504454047s
FAIL (2025-06-24 15:18:54 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
38.710079463 (29h51m38.593493084s) [seg_pos={0 0}, upload=23996]: Done begin upload segment.
Jun 24 15:02:38.710129041 (29h51m38.59354267s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.710138687 (29h51m38.593552328s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.710690489 (29h51m38.594104106s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.710713779 (29h51m38.594127417s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.711106997 (29h51m38.594520623s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.711531376 (29h51m38.594945005s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712074545 (29h51m38.59548818s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712237061 (29h51m38.595650701s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712469485 (29h51m38.595883111s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712638648 (29h51m38.596052285s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712933855 (29h51m38.596347485s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.713378835 (29h51m38.596792472s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.713813822 (29h51m38.597227453s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.713901007 (29h51m38.597314653s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.714349767 (29h51m38.597763401s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.714694647 (29h51m38.59810828s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.715041886 (29h51m38.598455519s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.715462120 (29h51m38.59887574s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:39.251183574 (29h51m39.134597192s) [upload=23996]: Next returned nil segment
Jun 24 15:02:39.251207849 (29h51m39.134621467s) [upload=23996]: Waiting for error group managing segments...
Jun 24 15:02:39.412884907 (29h51m39.296298523s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.417725225 (29h51m39.301138871s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.419058833 (29h51m39.302472446s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.428264821 (29h51m39.311678434s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.430916704 (29h51m39.314330317s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.441395180 (29h51m39.324808793s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.443411058 (29h51m39.326824675s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.453557685 (29h51m39.336971305s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.461322189 (29h51m39.344735804s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.473469757 (29h51m39.356883373s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.473537251 (29h51m39.356950869s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.486281273 (29h51m39.369694894s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.571776596 (29h51m39.455190214s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.653833016 (29h51m39.537246631s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.653863972 (29h51m39.537277598s) [seg_pos={0 0}, upload=23996]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:02:39.654154687 (29h51m39.537568301s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: expected piece hash; context canceled; EOF
Jun 24 15:02:39.654204520 (29h51m39.537618156s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ac3N4hMJ66bTx2dpHBDT6W5SEGnPNfwyonwmvAANR8mM8vvpY): protocol: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection
Jun 24 15:02:39.654312246 (29h51m39.537725915s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection
Jun 24 15:02:39.654354065 (29h51m39.537767695s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZgWv9RA5EpAHfxzBWLpTj8un93LPU8syqxM9gBQQomMP9MRwG): protocol: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection
Jun 24 15:02:39.654401800 (29h51m39.537815437s) [seg_pos={0 0}, upload=23996]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:02:39.654504756 (29h51m39.537918371s) [upload=23996]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:02:39.654510115 (29h51m39.537923741s) [upload=23996]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:02:39.810746072 (29h51m39.694159689s) [upload=23996]: Done uploading segments...
Jun 24 15:02:39.810767325 (29h51m39.694180941s) [upload=23996]: Upload finished. err: <nil>
Duration: 2.504485567s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
2.278500124s
FAIL (2025-06-24 15:18:48 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments...
Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil>
Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload
Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload
Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments...
Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false
Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment...
Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment.
Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment
Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments...
Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces.
Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled
Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments...
Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil>
Duration: 2.278528621s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
3.021901419s
FAIL (2025-06-24 15:18:32 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 3.021940719s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
876.31801ms
FAIL (2025-06-24 15:18:32 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 876.349418ms
edge
authservice
http
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
886.273089ms
FAIL (2025-06-24 15:18:25 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 886.304177ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
2.634273629s
FAIL (2025-06-24 15:18:19 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
38.710079463 (29h51m38.593493084s) [seg_pos={0 0}, upload=23996]: Done begin upload segment.
Jun 24 15:02:38.710129041 (29h51m38.59354267s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.710138687 (29h51m38.593552328s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.710690489 (29h51m38.594104106s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.710713779 (29h51m38.594127417s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.711106997 (29h51m38.594520623s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.711531376 (29h51m38.594945005s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712074545 (29h51m38.59548818s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712237061 (29h51m38.595650701s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712469485 (29h51m38.595883111s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712638648 (29h51m38.596052285s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712933855 (29h51m38.596347485s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.713378835 (29h51m38.596792472s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.713813822 (29h51m38.597227453s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.713901007 (29h51m38.597314653s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.714349767 (29h51m38.597763401s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.714694647 (29h51m38.59810828s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.715041886 (29h51m38.598455519s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.715462120 (29h51m38.59887574s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:39.251183574 (29h51m39.134597192s) [upload=23996]: Next returned nil segment
Jun 24 15:02:39.251207849 (29h51m39.134621467s) [upload=23996]: Waiting for error group managing segments...
Jun 24 15:02:39.412884907 (29h51m39.296298523s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.417725225 (29h51m39.301138871s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.419058833 (29h51m39.302472446s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.428264821 (29h51m39.311678434s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.430916704 (29h51m39.314330317s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.441395180 (29h51m39.324808793s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.443411058 (29h51m39.326824675s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.453557685 (29h51m39.336971305s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.461322189 (29h51m39.344735804s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.473469757 (29h51m39.356883373s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.473537251 (29h51m39.356950869s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.486281273 (29h51m39.369694894s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.571776596 (29h51m39.455190214s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.653833016 (29h51m39.537246631s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.653863972 (29h51m39.537277598s) [seg_pos={0 0}, upload=23996]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:02:39.654154687 (29h51m39.537568301s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: expected piece hash; context canceled; EOF
Jun 24 15:02:39.654204520 (29h51m39.537618156s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ac3N4hMJ66bTx2dpHBDT6W5SEGnPNfwyonwmvAANR8mM8vvpY): protocol: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection
Jun 24 15:02:39.654312246 (29h51m39.537725915s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection
Jun 24 15:02:39.654354065 (29h51m39.537767695s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZgWv9RA5EpAHfxzBWLpTj8un93LPU8syqxM9gBQQomMP9MRwG): protocol: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection
Jun 24 15:02:39.654401800 (29h51m39.537815437s) [seg_pos={0 0}, upload=23996]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:02:39.654504756 (29h51m39.537918371s) [upload=23996]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:02:39.654510115 (29h51m39.537923741s) [upload=23996]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:02:39.810746072 (29h51m39.694159689s) [upload=23996]: Done uploading segments...
Jun 24 15:02:39.810767325 (29h51m39.694180941s) [upload=23996]: Upload finished. err: <nil>
Duration: 2.634313604s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
2.309062583s
FAIL (2025-06-24 15:18:11 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments...
Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil>
Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload
Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload
Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments...
Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false
Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment...
Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment.
Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment
Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments...
Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces.
Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled
Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments...
Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil>
Duration: 2.309097199s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
2.274689989s
FAIL (2025-06-24 15:17:49 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments...
Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil>
Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload
Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload
Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments...
Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false
Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment...
Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment.
Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment
Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments...
Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces.
Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled
Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments...
Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil>
Duration: 2.274718735s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
917.304717ms
FAIL (2025-06-24 15:17:38 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 917.336153ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
3.67836304s
FAIL (2025-06-24 15:17:26 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 3.678385216s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
2.662729487s
FAIL (2025-06-24 15:17:21 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
38.710079463 (29h51m38.593493084s) [seg_pos={0 0}, upload=23996]: Done begin upload segment.
Jun 24 15:02:38.710129041 (29h51m38.59354267s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.710138687 (29h51m38.593552328s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.710690489 (29h51m38.594104106s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.710713779 (29h51m38.594127417s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.711106997 (29h51m38.594520623s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.711531376 (29h51m38.594945005s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712074545 (29h51m38.59548818s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712237061 (29h51m38.595650701s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712469485 (29h51m38.595883111s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712638648 (29h51m38.596052285s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712933855 (29h51m38.596347485s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.713378835 (29h51m38.596792472s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.713813822 (29h51m38.597227453s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.713901007 (29h51m38.597314653s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.714349767 (29h51m38.597763401s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.714694647 (29h51m38.59810828s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.715041886 (29h51m38.598455519s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.715462120 (29h51m38.59887574s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:39.251183574 (29h51m39.134597192s) [upload=23996]: Next returned nil segment
Jun 24 15:02:39.251207849 (29h51m39.134621467s) [upload=23996]: Waiting for error group managing segments...
Jun 24 15:02:39.412884907 (29h51m39.296298523s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.417725225 (29h51m39.301138871s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.419058833 (29h51m39.302472446s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.428264821 (29h51m39.311678434s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.430916704 (29h51m39.314330317s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.441395180 (29h51m39.324808793s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.443411058 (29h51m39.326824675s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.453557685 (29h51m39.336971305s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.461322189 (29h51m39.344735804s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.473469757 (29h51m39.356883373s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.473537251 (29h51m39.356950869s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.486281273 (29h51m39.369694894s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.571776596 (29h51m39.455190214s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.653833016 (29h51m39.537246631s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.653863972 (29h51m39.537277598s) [seg_pos={0 0}, upload=23996]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:02:39.654154687 (29h51m39.537568301s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: expected piece hash; context canceled; EOF
Jun 24 15:02:39.654204520 (29h51m39.537618156s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ac3N4hMJ66bTx2dpHBDT6W5SEGnPNfwyonwmvAANR8mM8vvpY): protocol: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection
Jun 24 15:02:39.654312246 (29h51m39.537725915s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection
Jun 24 15:02:39.654354065 (29h51m39.537767695s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZgWv9RA5EpAHfxzBWLpTj8un93LPU8syqxM9gBQQomMP9MRwG): protocol: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection
Jun 24 15:02:39.654401800 (29h51m39.537815437s) [seg_pos={0 0}, upload=23996]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:02:39.654504756 (29h51m39.537918371s) [upload=23996]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:02:39.654510115 (29h51m39.537923741s) [upload=23996]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:02:39.810746072 (29h51m39.694159689s) [upload=23996]: Done uploading segments...
Jun 24 15:02:39.810767325 (29h51m39.694180941s) [upload=23996]: Upload finished. err: <nil>
Duration: 2.662761403s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
876.084946ms
FAIL (2025-06-24 15:17:17 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 876.111243ms
edge
authservice
http
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
2.371534777s
FAIL (2025-06-24 15:16:43 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments...
Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil>
Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload
Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload
Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments...
Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false
Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment...
Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment.
Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment
Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments...
Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces.
Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled
Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments...
Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil>
Duration: 2.371586714s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
927.203883ms
FAIL (2025-06-24 15:16:39 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 927.229574ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
3.469056514s
FAIL (2025-06-24 15:16:34 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 3.469087258s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
2.837542178s
FAIL (2025-06-24 15:16:19 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
38.710079463 (29h51m38.593493084s) [seg_pos={0 0}, upload=23996]: Done begin upload segment.
Jun 24 15:02:38.710129041 (29h51m38.59354267s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.710138687 (29h51m38.593552328s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.710690489 (29h51m38.594104106s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.710713779 (29h51m38.594127417s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.711106997 (29h51m38.594520623s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.711531376 (29h51m38.594945005s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712074545 (29h51m38.59548818s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712237061 (29h51m38.595650701s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712469485 (29h51m38.595883111s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712638648 (29h51m38.596052285s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.712933855 (29h51m38.596347485s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.713378835 (29h51m38.596792472s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.713813822 (29h51m38.597227453s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.713901007 (29h51m38.597314653s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.714349767 (29h51m38.597763401s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.714694647 (29h51m38.59810828s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.715041886 (29h51m38.598455519s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:38.715462120 (29h51m38.59887574s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Uploading piece...
Jun 24 15:02:39.251183574 (29h51m39.134597192s) [upload=23996]: Next returned nil segment
Jun 24 15:02:39.251207849 (29h51m39.134621467s) [upload=23996]: Waiting for error group managing segments...
Jun 24 15:02:39.412884907 (29h51m39.296298523s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.417725225 (29h51m39.301138871s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.419058833 (29h51m39.302472446s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.428264821 (29h51m39.311678434s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.430916704 (29h51m39.314330317s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.441395180 (29h51m39.324808793s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.443411058 (29h51m39.326824675s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.453557685 (29h51m39.336971305s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.461322189 (29h51m39.344735804s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.473469757 (29h51m39.356883373s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.473537251 (29h51m39.356950869s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.486281273 (29h51m39.369694894s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.571776596 (29h51m39.455190214s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.653833016 (29h51m39.537246631s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil>
Jun 24 15:02:39.653863972 (29h51m39.537277598s) [seg_pos={0 0}, upload=23996]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:02:39.654154687 (29h51m39.537568301s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: expected piece hash; context canceled; EOF
Jun 24 15:02:39.654204520 (29h51m39.537618156s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ac3N4hMJ66bTx2dpHBDT6W5SEGnPNfwyonwmvAANR8mM8vvpY): protocol: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection
Jun 24 15:02:39.654312246 (29h51m39.537725915s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection
Jun 24 15:02:39.654354065 (29h51m39.537767695s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZgWv9RA5EpAHfxzBWLpTj8un93LPU8syqxM9gBQQomMP9MRwG): protocol: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection
Jun 24 15:02:39.654401800 (29h51m39.537815437s) [seg_pos={0 0}, upload=23996]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:02:39.654504756 (29h51m39.537918371s) [upload=23996]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:02:39.654510115 (29h51m39.537923741s) [upload=23996]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:02:39.810746072 (29h51m39.694159689s) [upload=23996]: Done uploading segments...
Jun 24 15:02:39.810767325 (29h51m39.694180941s) [upload=23996]: Upload finished. err: <nil>
Duration: 2.837589402s
core
uplink
us-select-1
us1-placement-20
|