Issue

• We can not create a Elasticsearch snapshot manually and this error is reported

"(Transport endpoint is not connected) 'status' : 500" 

• When I try to list the files in the path /usr/share/elasticsearch/snapshot/ this error is reported

cannot access /usr/share/elasticsearch/snapshot/: Transport endpoint is not connected

Environment

• CloudBees Jenkins Enterprise - AWS < 1.7.0

Resolution

This is because one or more worker are disconnected from the S3 Bucket. It is a known issue fixed in 1.7.0 version of CJE, let’s see which Elasticsearch worker is disconnected by checking each one by one:

Get the list of Elasticsearch workers

cje run list-workers | grep elasticsearch

Check if you can access to the S3 Bucket

dna connect worker-17
sudo ls -la /usr/share/elasticsearch/snapshot

One or more of the workers will fail when access to the s3 bucket with this error ls: cannot access /usr/share/elasticsearch/snapshot/: Transport endpoint is not connected, to mount it again execute these commands in that worker

kill -9 $(pgrep -x s3fs)
fusermount -u -z /usr/share/elasticsearch/snapshot
sudo /home/ec2-user/dna-config/s3fs-mount-cron

Tested product/plugin versions

• CloudBees Jenkins Enterprise 1.10.0