How to use multiline command in 'script:' with YAML?

Question

I have a repository that uses Travis CI, and in the .travis.yml there I have this line:

script:
- vim -Nu <(cat <<-EOF
  set nocompatible |
  filetype off
  EOF
  ) -c 'Script' > /dev/null

Sadly this doesn't work, as this is transformed into a single line and is executed like this:

vim -Nu <(cat <<-EOF set no compatible | filetype off | EOF ) -c 'Script' > /dev/null

This makes the EOF tag not working, as EOF needs to be in a single line. An alternative would be to just use normal quotes like this:

script:
- vim -Nu <(cat 'set nocompatible |
  filetype off
  ) -c 'Script' > /dev/null

Which works, and is fine, but I feel there must be a way to insert newlines into a .travis.yml. I have an alternative now, but I may not in the future. So how do you do it?

Anthon · Accepted Answer · 2017-08-11 07:03:30Z

75

In YAML you can specify newlines in a scalar by using "" quoting and escaping the newlines (\n), or, and that is more natural for your case, by using a literal style block scalar:

script:
- |
  vim -Nu <(cat <<-EOF
  set nocompatible |
  filetype off
  EOF
  ) -c 'Script' > /dev/null

This is a scalar starting with a line with a | (pipe symbol), followed by multiple lines for which the line-breaks are preserved.

The lines are normally indented (exception: a single top-level literal style block scalar).
After the | there can be modifiers: 1-9, used when your first line starts with spaces; +, - to influence stripping of final newlines (normally collapsed into one).

edited Aug 11, 2017 at 7:03

answered Aug 3, 2016 at 15:18

Anthon

78.3k35 gold badges207 silver badges290 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

serghei · Accepted Answer · 2023-07-21 23:14:35Z

67

Your question relates to the YAML language standard, which is used in the configuration files of many CI/CD systems, including Travis CI, GitLab CI, GitHub Actions, and others. However, it's worth noting that not all CI/CD systems support the YAML standard fully, and they might also add their own extensions.

YAML has special block styles for handling line breaks. In your case, the folded block style, denoted by the “>” symbol might be useful. It allows long lines to be broken up for readability while retaining the semantics of the original long line. Here's an example:

script:
    - >
      valgrind
      --read-var-info=yes
      --error-exitcode=1
      --fullpath-after=
      --track-origins=yes
      --leak-check=full
      --num-callers=20
      --suppressions=$(pwd)/tests/zephir_parser.3.7.0.sup
      $(phpenv which php)
          -d variables_order=EGPCS
          run-tests.php
              -p $(which php)
              -d extension=$(pwd)/modules/zephir_parser.so
              -d variables_order=EGPCS
              -g "FAIL,XFAIL,BORK,WARN,LEAK,SKIP"
              --offline
              --show-diff
              --set-timeout 120

In this style, each line break is replaced with a space. Indentation in each line will be ignored. A line break will be inserted at the end.

Note: The backslash “\” must not be used here, as it will be treated as a line continuation. Thus, the following example is wrong:

script:
    - >
      valgrind \               # this line wrong
      --read-var-info=yes

Otherwise it will be treated as valgrind \ --read-var-info=yes.

However, it's important to note that different CI/CD systems may have their peculiarities in processing configuration files. For instance, GitHub Actions uses a line folding style (“>-”), which allows long lines to be placed on multiple lines, but processes them as a single line. Here's an example:

  - name: Configure (x64 Debug)
    run: >-
      cmake
      -S .
      -Bbuild
      -DCMAKE_BUILD_TYPE=Debug
      -DCPPCHECK=ON
      -DWARNINGS_AS_ERRORS=ON
      -DCMAKE_INSTALL_PREFIX=/opt/my-program
      -DCMAKE_EXPORT_COMPILE_COMMANDS=ON

In this case, after processing, all lines are concatenated into one.

Note: With line folding lines are left aligned, using the same indentation at the beginning of the line.

edited Jul 21, 2023 at 23:14

answered Oct 30, 2017 at 15:01

serghei

3,4312 gold badges35 silver badges50 bronze badges

3 Comments

Chaim Eliyah Over a year ago

Can you throw a pipe in the middle of this? Or would that be bad?

Anthon Over a year ago

@ChaimEliyah YAML doesn't interpret the contents of the scalar beyond the newline after the first pipe, everything else is take "as is" (until you dedent).

Minh Nghĩa Over a year ago

I use \ as usual, and it breaks. Why isn't \ required in this case?

Shivam Anand · Accepted Answer · 2021-08-05 15:58:58Z

11

What you are looking for is |. Here is a very clear example to refer to.

   build:
     commands:
       - echo "${CODEBUILD_BUILD_ARN}"
       - |
         if expr "${CODEBUILD_BUILD_ARN}" : ".*build/MyProjectDev-" >/dev/null; then
           yarn run build-dev;
         fi
       - |
         if expr "${CODEBUILD_BUILD_ARN}" : ".*build/MyProject-" >/dev/null; then
           yarn run build-prod;
         fi

answered Aug 5, 2021 at 15:58

Shivam Anand

1,6811 gold badge15 silver badges23 bronze badges

1 Comment

joanis Over a year ago

This seems to repeat other answers, but thanks for showing how to use this syntax in a larger context, it's quite helpful.

Collectives™ on Stack Overflow

How to use multiline command in 'script:' with YAML?

3 Answers 3

Comments

3 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

3 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related