Access HTTP response data saved in a variable with colon

Question

I'm using Google Cloud Workflow to call via http.get a CloudRun app that returns a XML document that has been converted to json, the below json gets successfully returned to Workflow in Step 2 which contains the converted XML to json in the body.

{
  "body": {
    "ResponseMessage": {
      "@xmlns": "http://someurl.com/services",
      "Response": {
        "@xmlns:a": "http://someurl.com/data",
        "@xmlns:i": "http://www.w3.org/2001/XMLSchema-instance",
        "a:ReferenceNumber": {
          "@i:nil": "true"
        },
        "a:DateTime": "2023-01-01T00:17:38+0000",
        "a:TransactionId": "154200432",
        "a:Environment": "Development",
        "a:RequestDateTime": "2023-01-01T11:17:39",            
        }
    },
    "code": 200,
    "headers": {
        "Alt-Svc": "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\"",
        "Content-Length": "1601",
        "Content-Type": "application/json",
        "Date": "Sun, 01 Jan 2023 00:17:39 GMT",
        "Server": "Google Frontend",
        "X-Cloud-Trace-Context": "931754ab82102397eb07775171485850"
    }
  }
}

The full yaml of the workflow is below and without step3/step4 it works. In step3 I try to access an element in the json which is returned from step2 as per https://cloud.google.com/workflows/docs/http-requests#access-data

main:
  params: [args]
  steps:
    - step1:
        assign:
          - searchURL: ${"https://myfunction.a.run.app/search/" + args.type + "/" + args.serial}
    - step2:
        call: http.get
        args:
          url: ${searchURL}
        result: s2result
    - step3:
        assign:       
          - resubmitURL: '${https://myfunction.a.run.app/resubmit/" + ${s2result.body.ResponseMessage.Response.a:TransactionId} }'
    - step4:
        call: http.get
        args:
          url: ${resubmitURL}
        result: s4result
    - returnOutput:
        return: ${s4result}

However due to the colon : in the field I'm trying to access there are yaml parsing errors when I attempt to save another variable assignment. How can I access a HTTP response data saved in a variable when there are colons in the property field.

The errors in the console are similar too

    Could not deploy workflow: main.yaml:14:25: parse error: in workflow 'main', step 'step3': token recognition error at: ':'
          - resubmitURL: '${"https://myfunction.a.run.app/resubmit/" + ${s2result.body.ResponseMessage.Response.a:TransactionId}'
                        ^

main.yaml:14:25: parse error: in workflow 'main', step 'step3': mismatched input '+' expecting {'[', LOGICAL_NOT, '(', '-', TRUE, FALSE, NULL, NUM_FLOAT, NUM_INT, STRING, IDENTIFIER}
          - resubmitURL: '${"https://myfunction.a.run.app/resubmit/" + ${s2result.body.ResponseMessage.Response.a:TransactionId}'

Are you saying you have broken YAML and trying to figure out how to parse it? Or are you generating YAML and trying to figure out how to escape values? — Evert
– Evert, Commented Jan 1, 2023 at 4:21
@Evert I've edited the question to include more information. I'm attempting to define a workflow in yaml or json that can join a string ( a URL) with the body HTTP response json element that contains a colon e.g ${myvar.body.foo.bar.a:TransactionId} — LJT
– LJT, Commented Jan 1, 2023 at 7:01
@LJT As : is not one of those characters, it is obvious from the error line that it is expecting certain symbols. Therefore, try something like $s2result.body.ResponseMessage.Response["a:TransactionId"] as an alternative. Although I realise this is outdated syntax, please give the above a try for confirmation so we may narrow down our search for a solution. — Rohit Kharche
– Rohit Kharche, Commented Jan 1, 2023 at 7:12
Thanks @RohitKharche but that didn't work unfortunately, the error is similar. — LJT
– LJT, Commented Jan 1, 2023 at 7:31
So you are getting error on : I think you are getting because you are using extra curly brackets, this time try this - resubmitURL: "https://myfunction.a.run.app/resubmit/${s2result.body.ResponseMessage.Response['a:TransactionId']}" . — Rohit Kharche
– Rohit Kharche, Commented Jan 1, 2023 at 8:15

Kris Braun · Accepted Answer · 2023-01-03 16:04:14Z

2

Two techniques are required to reference map keys with special characters like this:

As recommended in the documentation, all expressions should be wrapped in single quotes to avoid YAML parsing errors (i.e. '${...}').
When referencing keys with special characters, you can use array notation to wrap the key name in quotes (i.e. var["KEY"]).

Together, it looks like this:

main:
    steps:
    - init:
        assign:
        - var:
            key: 
                "co:lon": bar
    - returnOutput:
        return: '${"foo" + var.key["co:lon"]}'

edited Jan 3, 2023 at 16:04

answered Jan 2, 2023 at 1:59

Kris Braun

1,46215 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

LJT Over a year ago

Thanks, but I'm using #1 and cloud.google.com/workflows/docs/http-requests#access-data PATH_TO_FIELD which contains the key with a colon to look-up the value, using #2 doesn't work. See comments on question.

Kris Braun Over a year ago

Ah, the whole value must be an expression (string interpolation is in private preview, so unless your project is in that preview, expressions cannot be used within a string, and expressions cannot be nested). So in your code, it should be - resubmitURL: '${"https://myfunction.a.run.app/resubmit/" + s2result.body.ResponseMessage.Response["a:TransactionId"]}'

LJT Over a year ago

That doesn't parse, it throws Could not deploy workflow: main.yaml:14:25: parse error: in workflow 'main', step 'step3': missing IDENTIFIER at '['

Kris Braun Over a year ago

I've updated the sample above to match your usage (nested map, string concatenation). I've confirmed it deploys and runs, and also confirmed the assignment in my comment above deploys (when s2result is properly defined).

LJT Over a year ago

Great, that works thanks for your assistance. I was incorrectly using two expressions, and not using square brackets. Used stackoverflow.com/questions/74972978/… and it worked. Worth noting that the comma is dropped between json elements when using square brackets it's not .Response.["a:TransactionId"] but .Response["a:TransactionId"]

r000bin · Accepted Answer · 2023-01-04 07:43:15Z

In your code you are using an expression inside an expression:

- resubmitURL: '**${**"https://myfunction.a.run.app/resubmit/" + **${**s2result.body.ResponseMessage.Response.a:TransactionId**}**'

In this sample from your error message your not even closing the expressions right.

If you pack everything into one expression and use the hint from Kris with the key, it should deploy:

- resubmitURL: '${"https://myfunction.a.run.app/resubmit/" + s2result.body.ResponseMessage.Response["a:TransactionId"]}'

Here is my full test case:

main:
  params: [args]
  steps:
    - init_assign:
        assign:
          - input: ${args}
          - s2result:
                body:
                    ResponseMessage:
                        Response:
                            "a:TransactionId": "Test"
          - resubmitURL: '${"https://myfunction.a.run.app/resubmit/" + s2result.body.ResponseMessage.Response["a:TransactionId"]}'

    - log1:
        call: sys.log
        args:
            text: ${resubmitURL}
            severity: INFO

With the log: 'https://myfunction.a.run.app/resubmit/Test'

Collectives™ on Stack Overflow

Access HTTP response data saved in a variable with colon

2 Answers 2

5 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related