Issue
- We can not create a Elasticsearch snapshot manually and this error is reported
"(Transport endpoint is not connected) 'status' : 500"
- When I try to list the files in the path
/usr/share/elasticsearch/snapshot/this error is reported
cannot access /usr/share/elasticsearch/snapshot/: Transport endpoint is not connected
Environment
- CloudBees Jenkins Enterprise - AWS < 1.7.0
Resolution
This is because one or more worker are disconnected from the S3 Bucket. It is a known issue fixed in 1.7.0 version of CJE, let’s see which Elasticsearch worker is disconnected by checking each one by one:
Get the list of Elasticsearch workers
cje run list-workers | grep elasticsearch
Check if you can access to the S3 Bucket
dna connect worker-17
sudo ls -la /usr/share/elasticsearch/snapshot
One or more of the workers will fail when access to the s3 bucket with this error
ls: cannot access /usr/share/elasticsearch/snapshot/: Transport endpoint is not connected, to mount it again execute these commands in that worker
kill -9 $(pgrep -x s3fs)
fusermount -u -z /usr/share/elasticsearch/snapshot
sudo /home/ec2-user/dna-config/s3fs-mount-cron
0 Comments