Execute in bash command stored in variable [duplicate]

Question

I'm trying to script a mailing using a curl API (this is the base API, in mine the html part is changed with "xmessage":

curl -s \
-X POST \
--user "$MJ_APIKEY_PUBLIC:$MJ_APIKEY_PRIVATE" \
https://api.mailjet.com/v3.1/send \
-H 'Content-Type: application/json' \
-d '{
    "Messages":[
            {
                    "From": {
                            "Email": "[email protected]",
                            "Name": "Mailjet Pilot"
                    },
                    "To": [
                            {
                                    "Email": "[email protected]",
                                    "Name": "passenger 1"
                            }
                    ],
                    "Subject": "Your email flight plan!",
                    "TextPart": "Dear passenger 1, welcome to Mailjet! May the delivery force be with you!",
                    "HTMLPart": "<h3>Dear passenger 1, welcome to <a href=\"https://www.mailjet.com/\">Mailjet</a>!</h3><br />May the delivery force be with you!",
                    "CustomCampaign": "SendAPI_campaign",
                    "DeduplicateCampaign": true
            }
    ]
}'

My script look like this :

...
message=$(cat ./message.txt)
message=${message//"xdate"/$courseDate}
message=${message//"xcoursecode"/$courseCode}
message=${message//"xsubtitle"/$subtitle}
message=${message//"\r"/""}
message=${message//"\r\n"/""}
message=${message//"\n"/""}
message=${message//"\""/"\\\""}
message=${message//"'"/"&apos;"}

mailJet=$(cat ./mailjet.txt) # containing my API as described as above
mailJet=${mailJet//"xmessage"/$message}

echo $mailJet
eval $mailJet

The command eval $mailJet does not work, but if I do a copy paste in the terminal of the echo $mailJet output my command works.

The eval $mailJet give the following error :

{"ErrorIdentifier":"5cce36c5-373c-48ca-90b8-2b6bfc5df526","ErrorCode":"mj-0031","StatusCode":400,"ErrorMessage":"Request payload contains not valid UTF-8 encoded characters"}

Something that partially worked, was to put directly the mailJet.txt content in the script, but I'm struggling to find the syntax to replace the xmessage by what's in $message.

Like this it did not worked :

...
message=$(cat ./message.txt)
message=${message//"xdate"/$courseDate}
message=${message//"xcoursecode"/$courseCode}
message=${message//"xsubtitle"/$subtitle}
message=${message//"\r"/""}
message=${message//"\r\n"/""}
message=${message//"\n"/""}
message=${message//"\""/"\\\""}
message=${message//"'"/"&apos;"}

curl -s \
-X POST \
--user "$MJ_APIKEY_PUBLIC:$MJ_APIKEY_PRIVATE" \
https://api.mailjet.com/v3.1/send \
-H 'Content-Type: application/json' \
-d '{
    "Messages":[
            {
                    "From": {
                            "Email": "[email protected]",
                            "Name": "Mailjet Pilot"
                    },
                    "To": [
                            {
                                    "Email": "[email protected]",
                                    "Name": "passenger 1"
                            }
                    ],
                    "Subject": "Your email flight plan!",
                    "TextPart": "Dear passenger 1, welcome to Mailjet! May the delivery force be with you!",
                    "HTMLPart": "$message",   ## neither like this : "HTMLPart": "'$message'",
                    "CustomCampaign": "SendAPI_campaign",
                    "DeduplicateCampaign": true
            }
    ]
}'

Whereas if I put any html stuff instead of $message in the curl API, the script run without any issue.

I'm stuck (and not a great bash coder or even coder at all).

Do you have jq? Using string substitution (and trying to figure out how to generate valid JSON yourself) is a bad idea in the first place. Use a real JSON generator built to the specification. — Charles Duffy
– Charles Duffy, Commented Mar 20, 2020 at 12:26
jq will do all the work needed to convert your string to JSON -- replacing literal characters with their escape-sequence equivalents; or, if you want to go the other direction, it can do that too. Also has built-in regex support, so there's no reason to do the xdate/xcoursecode/etc. replacement in bash. — Charles Duffy
– Charles Duffy, Commented Mar 20, 2020 at 12:29
Hi Charles, thank you for you suggests. I do not have jq, but it's not an issue, I could easily install it. I take a look at it, and I have to admit that I really do not have any idea on how to use it in my case. Then you write I'm trying to figure out how to generate valid JSON by myself. I'm just trying to use the model given on mailjet to use their API. I'm not sure the json is the issue as when I copy/paste the echo of the variable $mailJet it works, so this would means the issue is not the JSON, no ? — Processor
– Processor, Commented Mar 21, 2020 at 12:47
"Works" is a fuzzy concept. Something can look correct and not be correct; and it can be valid with some values but not others. The value of using a tool like jq is that its implementation is written to the letter of the JSON spec, so there's a firm guarantee that all possible values will work. — Charles Duffy
– Charles Duffy, Commented Mar 21, 2020 at 15:36
The answer by @GemTaylor is correct as far as it goes, and if you read the linked duplicate, you'll see similar answers there; the narrow fix is to change "HTMLPart": "$message" to "$HTMLPart": "'"$message"'". Thing is, that narrow fix still has you doing all the message=${message//"\""/"\\\""} munging by hand, and that code as-written is not nearly correct for all possible messages. (Being "correct for all possible messages" means escaping unicode; it's not even a realistic goal for a bash script to attempt without using built-to-purpose tools). — Charles Duffy
– Charles Duffy, Commented Mar 21, 2020 at 15:37

Gem Taylor · Accepted Answer · 2020-03-20 12:23:38Z

0

I think your problems start at

-d '{

This use of a single quote means that nothing in this section is interpreted by bash. "$message" later on will be treated as the text $ and message.

If this is the issue, then what you need to do is unquote around the variable names, but also double-quote arround them, so you write '"$message"' Or, if you need the double-quotes to appear inside the curl command "'"$message"'".

Note you can't have the ## comment either, but I assume you put that in for our benefit.

answered Mar 20, 2020 at 12:23

Gem Taylor

5,7351 gold badge11 silver badges34 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Charles Duffy Over a year ago

True insofar as it goes, but not generally considered best practice. One of the reasons the Answer Well-Asked Questions section of How to Answer advises closing duplicative questions rather than answering them is so we can have as many answers as possible in one place, voted/ranked against each other, edited in accordance with a relatively long history of commentary and review.

Charles Duffy Over a year ago

If you look at the current linked duplicate, you'll see this approach (of simple string substitution after changing quoting styles) already promoted there -- but also criticized, and alternatives (particularly, use of JSON-syntax-aware tooling such as jq) proposed.

Collectives™ on Stack Overflow

Execute in bash command stored in variable [duplicate]

1 Answer 1

2 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Linked

Related