how to use while loop to iterate a value in unix shell script

Question

I have a path location called "qe/performance" and want to copy "jmx" files to S3 Bucket in AWS. Once file copied, i want to use while loop to list down the converted files(.jtl) in the results folder into s3 bucket. If files are present (successful), it will update the ECS cluster and remove the ec2 instance to decrease the load and send message to the group that file has uploaded.

on the other hand, if converted files are not presented to the results folder into s3 bucket, i want to put "Timeoutthreshold for 10 minutes and use iteration and sleep for 60 seconds to wait for files to be uploaded to s3 bucket. If iteration period crosses the timeoutthreshold, it will send email notification that file hasnt uploaded....

#/bin/bash
TimeoutThreshold=10
holder="qe/performance/"
echo $holder
for files in $holder; do
  aws s3 cp $files s3://$JMeterFilesBucket/scripts/ --recursive
  i=0
    while [ true ]; do
      aws s3 ls s3://$JMeterFilesBucket/results/ --recursive |grep .jtl
      if [ "$?" -ne "0" ]; then \
        sleep 60
        iterator=$i + 1
        if [ iterator -eq "$TimeoutThreshold" ]; then
          aws sns publish --topic-arn ${Subscription} --message "files (HTML Reports/.jtl) failed to load to the s3 bucket: s3://$JMeterFilesBucket/results/";
        else
          sleep 60
        fi
        exit 0
      else [ "$?" -eq "0" ]; then \
        aws ecs update-service --cluster $Cluster --service $Service --desired-count $CURRENT_DESIRED_COUNT --region ${AWS::Region}
        echo "publishing SNS Notification for the files that have been uploaded successfully"
        aws sns publish --topic-arn ${Subscription} --message "files (HTML Reports/.jtl) have successfully loaded to the s3 bucket: s3://$JMeterFilesBucket/results/";
        break
      fi
    done
done

here is the result after running this command, it still continuing showing the list of files like this and break

Also no email notifications are going thru in either condition


qe/performance/

21:49:50
Completed 14.7 KiB/14.7 KiB (86.3 KiB/s) with 1 file(s) remaining upload: qe/performance/ContribAccl_Dev.jmx to s3://retqa-jmeterfiles/scripts/ContribAccl_Dev.jmx

21:49:52
2019-09-11 17:41:12 1460 results/ContribAccl_Dev.jmx.jtl

21:49:52
2019-09-06 03:16:32 1653 results/Sample13.jmx.jtl

21:49:52
2019-09-09 15:19:40 163 results/Sample15.jmx.jtl

21:49:52
2019-09-11 17:41:12 1460 results/ContribAccl_Dev.jmx.jtl

21:49:52
2019-09-06 03:16:32 1653 results/Sample13.jmx.jtl

21:49:52
2019-09-09 15:19:40 163 results/Sample15.jmx.jtl

21:49:52
2019-09-11 17:41:12 1460 results/ContribAccl_Dev.jmx.jtl

21:49:52
2019-09-06 03:16:32 1653 results/Sample13.jmx.jtl

21:49:52
2019-09-09 15:19:40 163 results/Sample15.jmx.jtl

21:49:52
2019-09-11 17:41:12 1460 results/ContribAccl_Dev.jmx.jtl

21:49:52
2019-09-06 03:16:32 1653 results/Sample13.jmx.jtl

21:49:52
2019-09-09 15:19:40 163 results/Sample15.jmx.jtl

21:49:54
2019-09-11 17:41:12 1460 results/ContribAccl_Dev.jmx.jtl

21:49:54
2019-09-06 03:16:32 1653 results/Sample13.jmx.jtl

21:49:54
2019-09-09 15:19:40 163 results/Sample15.jmx.jtl

21:49:54
2019-09-11 17:41:12 1460 results/ContribAccl_Dev.jmx.jtl

21:49:54
2019-09-06 03:16:32 1653 results/Sample13.jmx.jtl

21:49:54
2019-09-09 15:19:40 163 results/Sample15.jmx.jtl

while [ true ]; do should be while true; do. But that's not the problem. — Barmar
– Barmar, Commented Sep 13, 2019 at 3:40
You should start by pasting your script into shellcheck.net, it finds a bunch of problems that you should fix. — Barmar
– Barmar, Commented Sep 13, 2019 at 3:57

Barmar · Accepted Answer · 2019-09-13 03:53:16Z

1

You have a typo here:

#/bin/bash

That should begin with #!, not just #.

This is wrong:

else [ "$?" -eq "0" ];

You don't put a condition after else. If you want to do another test, you have to use elif.

But there's no need for another condition. This condition is just the opposite of the first condition, so just write else and it will execute the code.

      if [ "$?" -ne "0" ]; then
        sleep 60
        iterator=$i + 1
        if [ iterator -eq "$TimeoutThreshold" ]; then
          aws sns publish --topic-arn ${Subscription} --message "files (HTML Reports/.jtl) failed to load to the s3 bucket: s3://$JMeterFilesBucket/results/";
        else
          sleep 60
        fi
        exit 0
      else
        aws ecs update-service --cluster $Cluster --service $Service --desired-count $CURRENT_DESIRED_COUNT --region ${AWS::Region}
        echo "publishing SNS Notification for the files that have been uploaded successfully"
        aws sns publish --topic-arn ${Subscription} --message "files (HTML Reports/.jtl) have successfully loaded to the s3 bucket: s3://$JMeterFilesBucket/results/";
        break
      fi

There's also no need to escape the newline after then.

There also doesn't seem to be any point to the while loop. If the condition is true it will use exit to terminate the script, but if the condition is false it will use break to stop the loop. Either way, only the first iteration of the loop runs.

This is not the correct way to increment variable:

iterator=$i + 1

This tries to execute the statement + 1 with the environment variable iterator set to $i. But + 1 is not a valid command. You should write:

((iterator = $i + 1))

The double parenthese make it an arithmetic expression instead of an ordinary command.

Then you need to put $ before the iterator variable when you test it:

if [ "$iterator" -eq "$TimeoutThreshold" ]; then

However, $i is always 0 because you never increment that. If the loop didn't end immediately, you could use

((i++))

at the end of the loop.

edited Sep 13, 2019 at 3:53

answered Sep 13, 2019 at 3:45

Barmar

789k57 gold badges554 silver badges669 bronze badges

Sign up to request clarification or add additional context in comments.

10 Comments

Ankur1825 Over a year ago

Thank you so much @Barmar. It worked. So where would i use ((i++))

Barmar Over a year ago

At the bottom of the loop when you want to increment the counter.

Barmar Over a year ago

But like I said, your exit and break statements mean it's not actually looping. I don't really understand what you're trying to do with all those loops.

Ankur1825 Over a year ago

i use "exit 0" at the end of the loop

Barmar Over a year ago

It's not at the end of the loop, it's in if [ "$?" -ne "0" ];. And when $? == 0 you do break. Either way, the loop ends.

|

William Pursell · Accepted Answer · 2019-09-13 03:57:08Z

I just want to address the general construct of your if/else statement. If you get the syntax correct, you would write:

if [ "$?" -ne "0" ]; then 
  command_1
elif [ "$?" -eq "0" ]; then
  command_2   # This can never happen
else
  command_3
fi

but that is wrong. The second list of commands (the portion of the elif) can never happen. The reason is that if the command that precedes this fails, then command_1 will be executed and neither of command_2 or command_3. But if the command that precedes this does not fail, then [ "$?" -ne "0" ] will fail, setting $? to a non-zero value. Then [ "$?" -eq "0" ] will fail because the command executed by the if portion changed the value of $?. Don't write if/else like this.

Collectives™ on Stack Overflow

how to use while loop to iterate a value in unix shell script

2 Answers 2

10 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

10 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related