I have a file (pattern file) I would like to feed into grep via -f, and I would like to find matches in another file (search file) where a string starts with that given pattern. For example:
PATTERN_FILE
1234
qwerty
chicken
SEARCH_FILE
12345
543212345
qwerty
1fwf32sgww
chicken fingers
Given the above files, grep should return the following lines
12345
qwerty
chicken fingers
How would I do this?
You could prepend ^ to all lines in PATERN_FILE to pass to grep -f:
paste -d '^' /dev/null PATTERN_FILE | grep -f - SEARCH_FILE
Or sed 's/^/^/' PATTERN_FILE instead of paste.
Now, if the lines in PATTERN_FILE are meant to be fixed strings rather than basic regular expressions, then you'd also need to escape all the regex operators:
sed 's/[][$^*\\.]/\\&/g; s/^/^/' PATTERN_FILE | grep -f - SEARCH_FILE
Alternative: you can use awk rather than grep to do the matching:
awk 'NR==FNR{a[$0];next} {for(t in a) if(substr($0,1,length(t))==t){print;next}}' needles haystack
This avoids the overhead of the regexp engine in grep, but has awk's interpretation overhead; I'm not sure which comes out better.
Using Raku (formerly known as Perl_6)
~$ raku -ne 'BEGIN my @a = "/path/to/pattern_file.txt".IO.lines; \
.put if .starts-with( any @a );' search_file.txt
#OR
~$ raku -ne 'BEGIN my @a = "/path/to/pattern_file.txt".IO.lines; \
.put if .starts-with( [|] @a );' search_file.txt
Above are answers written in Raku, a member of the Perl-family of programming languages. Assuming here that the pattern_file.txt is meant to be fixed strings rather than basic regular expressions, Raku has string-matching functions like starts-with and ends-with. Raku also has Junctions like any, all, one, and none that can simplify this matching problem.
Above the -ne non-autoprinting command line flags are used, which reads input files linewise. In a BEGIN block the pattern_file.txt is read into @a array. In the body of the code, the input line is output if it starts with any element of @a (first answer). Alternatively (second answer), Raku's [ ] reduction meta-operator notation is used to conceptually insert a | OR operator between the elements of @a. The first and second answers give identical results.
Sample Input:
pattern_file.txt
1234
qwerty
chicken
search_file.txt
12345
543212345
qwerty
1fwf32sgww
chicken fingers
Sample Output:
12345
qwerty
chicken fingers
Note: It's tempting to think that a one Junction (or equivalent [^] reduction meta-operator) will accomplish the same task, but this only holds true if each line in patterns_file.txt is unique!
https://docs.raku.org/routine/starts-with
https://docs.raku.org/type/Junction
https://docs.raku.org/language/operators#Reduction_metaoperators
https://raku.org
One option to do this would be to modify the patterns file first, using a command such as the following to add a ^ character to the start of each line. This would instruct grep via regex to only match at the start of the line.
awk '{print "^" $0}' PATTERN_FILE
However this modifies the pattern file, and I would prefer a solution which does not alter the original list
-i inplace. You could awk '{print "^" $0}' needles | grep -f- haystack except if needles contains regexp operators, then it's more complicated as Stéphane noted -- awk can do those also, but not at easily as sed.