Regular Expression to extract php code partially (( array definition ))

@bwoebi It doesn't get parsed, I have provided an extended example.

I tested it thoroughly and it should work, don't hesitate to report a bug if you find one.

@bwoebi intentional or not, regex is about matching certain patterns. If I didn't write it to match ${"variable"} (for example) then it won't match. Basically I tried my best matching all possible cases, but since I'm not an advanced php coder, I forgot some cases (which I didn't even know).

@HamZa check out all the possible language: lxr.php.net/xref/PHP_TRUNK/Zend/zend_language_parser.y#282

@HamZa your effort deserves at least 250, i know this RegEx can be improved but currently just work on many cases. (( i will give 250 to bwoebi too ))

|

Community · Accepted Answer · 2017-05-23 12:24:58Z

21

Even when you asked for a regex, it works also with pure PHP. token_get_all is here the key function. For a regex check @HamZa's answer out.

The advantage here is that it is more dynamic than a regex. A regex has a static pattern, while with token_get_all, you can decide after every single token what to do. It even escapes single quotes and backslashes where necessary, what a regex wouldn't do.

Also, in regex, you have, even when commented, problems to imagine what it should do; what code does is much easier to understand when you look at PHP code.

$code = ' array(

  0  => "a",
  "a" => $GlobalScopeVar,
  "b" => array("nested"=>array(1,2,3)),  
  "c" => function() use (&$VAR) { return isset($VAR) ? "defined" : "undefined" ; },
  "string_literal",
  12345

); ';

$token = token_get_all("<?php ".$code);
$newcode = "";

$i = 0;
while (++$i < count($token)) { // enter into array; then start.
        if (is_array($token[$i]))
                $newcode .= $token[$i][1];
        else
                $newcode .= $token[$i];

        if ($token[$i] == "(") {
                $ending = ")";
                break;
        }
        if ($token[$i] == "[") {
                $ending = "]";
                break;
        }
}

// init variables
$escape = 0;
$wait_for_non_whitespace = 0;
$parenthesis_count = 0;
$entry = "";

// main loop
while (++$i < count($token)) {
        // don't match commas in func($a, $b)
        if ($token[$i] == "(" || $token[$i] == "{") // ( -> normal parenthesis; { -> closures
                $parenthesis_count++;
        if ($token[$i] == ")" || $token[$i] == "}")
                $parenthesis_count--;

        // begin new string after T_DOUBLE_ARROW
        if (!$escape && $wait_for_non_whitespace && (!is_array($token[$i]) || $token[$i][0] != T_WHITESPACE)) {
                $escape = 1;
                $wait_for_non_whitespace = 0;
                $entry .= "'";
        }

        // here is a T_DOUBLE_ARROW, there will be a string after this
        if (is_array($token[$i]) && $token[$i][0] == T_DOUBLE_ARROW && !$escape) {
                $wait_for_non_whitespace = 1;
        }

        // entry ended: comma reached
        if (!$parenthesis_count && $token[$i] == "," || ($parenthesis_count == -1 && $token[$i] == ")" && $ending == ")") || ($ending == "]" && $token[$i] == "]")) {
                // go back to the first non-whitespace
                $whitespaces = "";
                if ($parenthesis_count == -1 || ($ending == "]" && $token[$i] == "]")) {
                        $cut_at = strlen($entry);
                        while ($cut_at && ord($entry[--$cut_at]) <= 0x20); // 0x20 == " "
                        $whitespaces = substr($entry, $cut_at + 1, strlen($entry));
                        $entry = substr($entry, 0, $cut_at + 1);
                }

                // $escape == true means: there was somewhere a T_DOUBLE_ARROW
                if ($escape) {
                        $escape = 0;
                        $newcode .= $entry."'";
                } else {
                        $newcode .= "'".addcslashes($entry, "'\\")."'";
                }

                $newcode .= $whitespaces.($parenthesis_count?")":(($ending == "]" && $token[$i] == "]")?"]":","));

                // reset
                $entry = "";
        } else {
                // add actual token to $entry
                if (is_array($token[$i])) {
                        $addChar = $token[$i][1];
                } else {
                        $addChar = $token[$i];
                }

                if ($entry == "" && $token[$i][0] == T_WHITESPACE) {
                        $newcode .= $addChar;
                } else {
                        $entry .= $escape?str_replace(array("'", "\\"), array("\\'", "\\\\"), $addChar):$addChar;
                }
        }
}

//append remaining chars like whitespaces or ;
$newcode .= $entry;

print $newcode;

Demo at: http://3v4l.org/qe4Q1

Should output:

array(

  0  => '"a"',
  "a" => '$GlobalScopeVar',
  "b" => 'array("nested"=>array(1,2,3))',  
  "c" => 'function() use (&$VAR) { return isset($VAR) ? "defined" : "undefined" ; }',
  '"string_literal"',
  '12345'

)

You can, to get the array's data, print_r(eval("return $newcode;")); to get the entries of the array:

Array
(
    [0] => "a"
    [a] => $GlobalScopeVar
    [b] => array("nested"=>array(1,2,3))
    [c] => function() use (&$VAR) { return isset($VAR) ? "defined" : "undefined" ; }
    [1] => "string_literal"
    [2] => 12345
)

edited May 23, 2017 at 12:24

CommunityBot

11 silver badge

answered Jun 16, 2013 at 15:36

bwoebi

23.8k5 gold badges64 silver badges81 bronze badges

9 Comments

hey!!, i like the magic* you do here, i don't care if this is not a RegEx solution cuz it actually works, in fact, love the way you go beyond the required solution (( RegEx )) and give just what i need... i tested it and apparently there are no bugs*, so, congrats!! you awarded +250 for this beauty piece of code. (( yes, you should wait, cuz im waiting 23hours to give +250 points to Hamza first ))

@iim.hlk no, magic, just processing an array returned by token_get_all() ;-P I'll wait for the bounty ;-)

iam trying to start a new bunty of 250 to "pay to you what is yours" but only gives me the choice of 500 points :c

@iim.hlk didn't you know that, if you give another bounty on the same question, you have to double the bounty?

@iim.hlk now: what'll you do? give a bounty (I really wrote this code to get the bounty :|) or not?

|

Casimir et Hippolyte · Accepted Answer · 2015-11-25 11:23:10Z

The clean way to do this is obviously to use the tokenizer (but keep in mind that the tokenizer alone doesn't solve the problem).

For the challenge, I purpose a regex approach.

The idea is not to describe the PHP syntax, but more to describe it in a negative way (in other words, I describe only basic and needed PHP structures to obtain the result). The advantage of this basic description is to deal with more complex objects than functions, strings, integers or booleans. The result is a more flexible pattern that can deal for example with multi/single line comments, heredoc/nowdoc syntaxes:

<pre><?php

$code=' array(
  0   => "a",
  "a" => $GlobalScopeVar,
  "b" => array("nested"=>array(1,2,3)),  
  "c" => function() use (&$VAR) { return isset($VAR) ? "defined" : "undefined" ; },
); ';

$pattern = <<<'EOD'
~
# elements
(?(DEFINE)
    # comments
    (?<comMulti> /\* .*? (?:\*/|\z) )                                              # multiline comment
    (?<comInlin> (?://|\#) \N* $ )                                                 # inline comment
    (?<comments> \g<comMulti> | \g<comInlin> )

    # strings
    (?<strDQ> " (?>[^"\\]+|\\.)* ")                                                # double quote string
    (?<strSQ> ' (?>[^'\\]+|\\.)* ')                                                # single quote string
    (?<strHND> <<<(["']?)([a-zA-Z]\w*)\g{-2} (?>\R \N*)*? \R \g{-1} ;? (?=\R|$) )  # heredoc and nowdoc syntax
    (?<string> \g<strDQ> | \g<strSQ> | \g<strHND> )

    # brackets
    (?<braCrl> { (?> \g<nobracket> | \g<brackets> )* } )
    (?<braRnd> \( (?> \g<nobracket> | \g<brackets> )* \) )
    (?<braSqr> \[ (?> \g<nobracket> | \g<brackets> )* ] )
    (?<brackets> \g<braCrl> | \g<braRnd> | \g<braSqr> )

    # nobracket: content between brackets except other brackets
    (?<nobracket> (?> [^][)(}{"'</\#]+ | \g<comments> | / | \g<string> | <+ )+ )

    # ignored elements
    (?<s> \s+ | \g<comments> )
)

# array components
(?(DEFINE)    
    # key
    (?<key> [0-9]+ | \g<string> )

    # value
    (?<value> (?> [^][)(}{"'</\#,\s]+ | \g<s> | / | \g<string> | <+ | \g<brackets> )+? (?=\g<s>*[,)]) )
)
(?J)
(?: \G (?!\A)(?<!\)) | array \g<s>* \( ) \g<s>* \K

    (?: (?<key> \g<key> ) \g<s>* => \g<s>* )? (?<value> \g<value> ) \g<s>* (?:,|,?\g<s>*(?<stop> \) ))
~xsm
EOD;


if (preg_match_all($pattern, $code, $m, PREG_SET_ORDER)) {
    foreach($m as $v) {
        echo "\n<strong>Whole match:</strong> " . $v[0]
           . "\n<strong>Key</strong>:\t" . $v['key']
           . "\n<strong>Value</strong>:\t" . $v['value'] . "\n";
        if (isset($v['stop']))
            echo "\n<strong>done</strong>\n\n"; 

    }
}

hey bro, i have a little problem with this solution just need a little fix i think, can check it please : ideone.com/FKrPxS, maybe there is a problem with the empty($v['stop']) conditional, thanks for your time.

zx81 · Accepted Answer · 2014-04-12 04:07:21Z

3

Here is what you asked for, very compact. Please let me know if you'd like any tweaks.

THE CODE (you can run this straight in php)

$code=' array(
  0  => "a",
 "a" => $GlobalScopeVar,
 "b" => array("nested"=>array(1,2,3)),  
 "c" => function() use (&$VAR) { return isset($VAR) ? "defined" : "undefined" ; },

); ';

$regex = "~(?xm)
^[\s'\"]*([^'\"\s]+)['\"\s]*
=>\s*+
(.*?)\s*,?\s*$~";

if(preg_match_all($regex,$code,$matches,PREG_SET_ORDER)) {
    $array=array();
    foreach($matches as $match) {
        $array[$match[1]] = $match[2];
    }

    echo "<pre>";
    print_r($array);
    echo "</pre>";

} // END IF

THE OUTPUT

Array
(
    [0] => "a"
    [a] => $GlobalScopeVar
    [b] => array("nested"=>array(1,2,3))
    [c] => function() use (&$VAR) { return isset($VAR) ? "defined" : "undefined" ; }
)

$array contains your array.

You like?

Please let me know if you have any questions or require tweaks. :)

answered Apr 12, 2014 at 4:07

zx81

42k10 gold badges92 silver badges106 bronze badges

1 Comment

hey bro! i didn't see this answer until now, looks really good, in fact i always love compact the code (( do more with few lines )), currently i can't test this, but i really appreciate your effort, thanks :)

HamZa · Accepted Answer · 2013-06-14 22:40:45Z

2

Just for this situation:

$code=' array(

  0=>"a",
  "a"=>$GlobalScopeVar,
  "b"=>array("nested"=>array(1,2,3)),  
  "c"=>function() use (&$VAR) { return isset($VAR) ? "defined" : "undefined" ; },

); ';

preg_match_all('#\s*(.*?)\s*=>\s*(.*?)\s*,?\s*$#m', $code, $m);
$array = array_combine($m[1], $m[2]);
print_r($array);

Output:

Array
(
    [0] => "a"
    ["a"] => $GlobalScopeVar
    ["b"] => array("nested"=>array(1,2,3))
    ["c"] => function() use (&$VAR) { return isset($VAR) ? "defined" : "undefined" ; }
)

edited Jun 14, 2013 at 22:40

answered Jun 14, 2013 at 22:30

HamZa

15k11 gold badges56 silver badges75 bronze badges

4 Comments

why you say "Just for this situation" ??, this regex can't extract any kind of properly structured array??

@iim.hlk It will fail for something like this. Basically each "element" needs to be on a new line in order for this to succeed.

Geez!, sounds dirty but i can offer 400 of my rep to get that magic waterproof invincible regex... think about it, with +6000 of rep you will catch ALL the ladies (( and ocasionally a JB )) :$