I've tried using sed for this. I've tried putting the lines of interest in variables as well.
I have two examples I want to achieve for now. Lets say I have thousands of urls in a file called links.txt, here are the first three:
EDIT: I added another site to show domain for real-world example?
https://site.com/category/feed/
https://site2.org/feed/
https://site3.net/science/astronomy/feed/
https://feed.site4.info/market/feed/news.xml
and I paste these variables in the terminal:
TAG='<outline type="rss" title= text= version="RSS" xmlUrl= htmlUrl=/>'
NAMES=$(sed "s/https\:\/\///g;s/[\/].*//;s/\..*//g" links.txt)
XMLS=$(sed "s/.*/xmlUrl=\"&\"/" links.txt)
HTMLS=$(sed "s/.*/htmlUrl=\"&\"/g" links.txt)
How can I use this kind of strategy:
while IFS= read -r line; do echo "$line"
to take the "stream" of the first three lines in the links.txt file, suffix the next stream with the variables on the same line numbers, to create the name number of lines of populated $TAG of each link, and >> COMBINED.txt?
This is the result I want:
<outline type="rss" title="site" text="site" version="RSS" xmlUrl="https://site.com/category/feed/" htmlUrl="https://site.com/"/>
<outline type="rss" title="site2" text="site2" version="RSS" xmlUrl="https://site2.org/feed/" htmlUrl="https://site2.org/"/>
<outline type="rss" title="site3" text="site3" version="RSS" xmlUrl="https://site3.net/science/astronomy/feed/" htmlUrl="https://site3.net/"/>
<outline type="rss" title="site4" text="site4" version="RSS" xmlUrl="https://feed.site4.info/market/feed/news.xml" htmlUrl="https://site4.info/"/>
I've tried several attempts, things like this and many others
TAG1='<outline type="rss"'
TAG2='version="RSS"'
TAG3='\/>'
echo "$XMLS" >> xmlurls.txt
sed "s/.*/$TAG1 & $TAG2 /" < xmlurls.txt | sed "s/[ \t]*$/$TAG3/" >> COMBINED.txt
I’ve tried modifying the variables, escaping slashes but frequently get the error of unterminated s.
Here's another example of what I want to do with this kind of strategy: one file has a couple of dozen lines, here are the first few lines
llvm-cfi-verify
llvm-config
llvm-cov
llvm-cvtres
llvm-cxxdump
llvm-cxxfilt
llvm-diff
llvm-dis
llvm-dlltool
llvm-dwarfdump
llvm-dwp
I applied the following below:
while IFS= read -r line; do echo "$line" | sed "s/.*/ --slave \/usr\/bin\/"$line"\\t\\t\\t\\t& \\t\\t\\t\\t\/usr\/bin\/"$line"-\${version}/g"; done < llvm.txt >> COMBINED
Result:
--slave /usr/bin/llvm-cfi-verify llvm-cfi-verify /usr/bin/llvm-cfi-verify-${version}
--slave /usr/bin/llvm-config llvm-config /usr/bin/llvm-config-${version}
--slave /usr/bin/llvm-cov llvm-cov /usr/bin/llvm-cov-${version}
--slave /usr/bin/llvm-cvtres llvm-cvtres /usr/bin/llvm-cvtres-${version}
--slave /usr/bin/llvm-cxxdump llvm-cxxdump /usr/bin/llvm-cxxdump-${version}
--slave /usr/bin/llvm-cxxfilt llvm-cxxfilt /usr/bin/llvm-cxxfilt-${version}
--slave /usr/bin/llvm-diff llvm-diff /usr/bin/llvm-diff-${version}
--slave /usr/bin/llvm-dis llvm-dis /usr/bin/llvm-dis-${version}
--slave /usr/bin/llvm-dlltool llvm-dlltool /usr/bin/llvm-dlltool-${version}
--slave /usr/bin/llvm-dwarfdump llvm-dwarfdump /usr/bin/llvm-dwarfdump-${version}
--slave /usr/bin/llvm-dwp llvm-dwp /usr/bin/llvm-dwp-${version}
the tab spacing wasn't pretty, but at least it's very close to what I want. Additionally, I am only applying one modification to one file, and my goal is to apply this sort of thing to chain multiple files/variable "streams" into a combined file.:
I'm on debian 13 with gnu sed, but I can also test this on alpine, fedora, void, opensuse, etc.. if needed.
Thanks for reading. I tried reading the following, it was difficult to find this kind of question. Maybe the keywords I use in google are incorrect.
- How to append the lines of a file to the end of the lines of other file?
- https://stackoverflow.com/questions/16644645/how-to-read-a-line-from-each-of-several-files-in-each-iteration-of-a-single-loop/16649390#16649390
- https://stackoverflow.com/questions/10929453/read-a-file-line-by-line-assigning-the-value-to-a-variable/10929511#10929511
- https://stackoverflow.com/questions/1521462/looping-through-the-content-of-a-file-in-bash
- https://tecadmin.net/looping-through-the-content-of-a-file-in-bash/
- https://stackoverflow.com/questions/4128235/what-is-the-exact-meaning-of-ifs-n/66942306#66942306
- https://stackoverflow.com/questions/9591744/how-to-add-to-the-end-of-lines-containing-a-pattern-with-sed-or-awk
- https://learnbyexample.github.io/learn_gnused/adding-content-from-file.html
- https://stackoverflow.com/questions/38263274/add-prefix-to-each-word-of-each-line-in-bash
- https://stackoverflow.com/questions/13586349/how-can-i-prepend-a-string-to-the-beginning-of-each-line-in-a-file
- How can I iterate through each line of a file using each line to replace a string in other files, creating a new file each time
EDIT: @markp-fuso, I've been able add/remove the "https://", or add an "s" to "http://" and create new text files without that prefix, but in the end, I want it like that. Thanks everyone, all useful information. I guess I need to learn perl basics now.
echo 'xmlUrl="https://site.com/category/feed/"' | sed 's/.*/<outline type="rss" & version="RSS" /' | sed 's/[ \t]*$/\/>/'site3.netbut the expected output has axmlUrlentry with a domain ofsite3.com(.netvs.com); if that's not a typo then please update the question with an explanation on when/how the domain should be modifiedsite.com); in your real world data can the domain contain more than 2 such strings (eg,a.nother.site.com)? can your real world data include ip addresses and/or ports (eg,5.6.7.8,4.5.6.7:260)? if any of these are possible then please update the question to a) show such examples and b) explain what should be assigned to thetitleandtextattributes in the expected result