parsing out string in csv file type using perl

Question

I am using perl's Tie::File to parse through a .csv file and matching for a specific string, its actually the first string/header on the file.

The problem I am having might be my input file type. The tool that exports the data file can export in .csv or text which I tried and tested both.

Somehow, I am still NOT getting the match. My problem could be two-fold: (1) my regex is wrong and/or (2) the file type.

Sample file header/string (if I cat the file):

??Global  Mail_Date.Dat

Sample file header/string (if I open up in editor, apple's TextEdit.app)

Global  Mail_Date.Dat

Here's the octal dump:

0000000 377 376   G  \0   l  \0   o  \0   b  \0   a  \0   l  \0      \0
        feff 0047 006c 006f 0062 0061 006c 0020
0000020      \0   M  \0   a  \0   i  \0   l  \0   _  \0   D  \0   a  \0
        0020 004d 0061 0069 006c 005f 0044 0061
0000040   t  \0   e  \0   .  \0   D  \0   a  \0   t  \0  \r  \0  \n  \0
        0074 0065 002e 0044 0061 0074 000d 000a

Obviously, doing an os cat shows a leading ?? on the string.

Code:

use strict;
use warnings;
use Tie::File;
use File::Copy;

    for (@ARGV) {
        tie my @lines, "Tie::File", $_;             
        #shift @lines if $lines[0] =~ /^Global/;
        if ($lines[0] =~ /^Global/) 
        {
             print "We have a match, remove the line ..";
             #shift @lines if $lines[0] =~ /^Global/;
             untie @lines; 
        }
        else
        { 
             print "No match found. Exit";
        }

}

The ?? could be a unicode BOM which TextEdit sees/skips. cat is just a plain "dump to output" and doesn't process the text in any way. — Marc B
– Marc B, Commented Oct 11, 2011 at 20:06
so, my regex should work then right? is there a better way to write it? — jdamae
– jdamae, Commented Oct 11, 2011 at 20:23
No, because Perl will also suck in the BOM. Your regex is looking for "Global" as the very first thing on a line - which it isn't - there's two unknown characters before it, so the regex never matches. — Marc B
– Marc B, Commented Oct 11, 2011 at 20:24
It seems like just removing the ^ in /^Global/ would fix it, unless you expect the word "Global" on other lines. Maybe something like /^.{0,2}Global/. — Chriszuma
– Chriszuma, Commented Oct 11, 2011 at 21:00
@Chriszuma - thanks. i did try both your suggestions and still no luck. I did add the octal dump (see edit). I can't figure it out yet. — jdamae
– jdamae, Commented Oct 12, 2011 at 1:35

ErikR · Accepted Answer · 2011-10-12 02:41:51Z

1

It looks like your file is encoded in utf16.

Try something like this:

binmode STDIN, ':encoding(UTF-16LE)';
while (<STDIN>) {
  if (m/Global/) {  # see note
    print "Matched Global on line $.\n";
  }
}

If you get a match then at least we know the encoding is correct.

To compensate for the BOM code-point, you could read in a single character after the binmode call:

binmode STDIN, ':encodeing(UTF-16LE)';
read(STDIN, my $buf, 1);
while (<STDIN>) {
  if (m/^Global/) { ... }
}

answered Oct 12, 2011 at 2:41

ErikR

52.2k9 gold badges78 silver badges136 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

David W. · Accepted Answer · 2011-10-12 02:52:06Z

1

I'm looking at the octal dump and notice the null character between each of your regular characters. That is, it's G-\0-l-\0-o-\0-b-\0-a-\0-l-\0 and not G-l-o-b-a-l. This means your file is not in ASCII text. Is this in UTF8 or UTF16? If so, you have to use the encoding function when you open the file in Perl:

open(my $fh, "<:encoding(UTF-16)", $fileName)
    or die qq(Can't open file "$fileName" for reading);

If this is a csv file, you should try the Text::CSV::Encoded module. This will help you parse your CSV file.

answered Oct 12, 2011 at 2:52

David W.

107k40 gold badges224 silver badges349 bronze badges

Collectives™ on Stack Overflow

parsing out string in csv file type using perl

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related