Home Original page

PHP :: Bug #35655 :: Missing end-of-line token

Bug #35655 Missing end-of-line token
Submitted: 2005-12-13 10:52 UTC Modified: 2005-12-13 22:45 UTC
From: tomas_matousek at hotmail dot com Assigned: iliaa (profile)
Status: Closed Package: Scripting Engine problem
PHP Version: 5CVS-2005-12-13 (snap) OS: *
Private report: No CVE-ID: None

 [2005-12-13 10:52 UTC] tomas_matousek at hotmail dot com

Description:
------------
PHP lexer omits end-of-line characters following T_END_HEREDOC token. This also causes incorrect output of highlight_string() function.

Reproduce code:
---------------
$code = '
<? 
  $x = <<<EOT
some string    
EOT
  $y = 2;
?>';

display_tokens(token_get_all($code));
highlight_string($code);



function display_tokens($tokens)
{
  foreach ($tokens as $token)
  { 
    if (is_array($token))
    {
      echo token_name($token[0]),"\n";
      echo $token[0]," '",htmlentities(addcslashes($token[1],"\n\r\t")),"'\n";
    }
    else
    {
      echo "    '",htmlentities($token),"'\n";
    }  
  }
}

Expected result:
----------------
T_INLINE_HTML
311 '\r\n'
T_OPEN_TAG
367 '<?'
T_WHITESPACE
370 ' \r\n  '
T_VARIABLE
309 '$x'
T_WHITESPACE
370 ' '
    '='
T_WHITESPACE
370 ' '
T_START_HEREDOC
371 '<<<EOT\r\n'
T_STRING
307 'some'
T_ENCAPSED_AND_WHITESPACE
314 ' '
T_STRING
307 'string'
T_ENCAPSED_AND_WHITESPACE
314 '    \r\n'
T_END_HEREDOC
372 'EOT'
T_WHITESPACE
370 '\r\n  '      // '\r\n' is missing in actual result
T_VARIABLE
309 '$y'
T_WHITESPACE
370 ' '
    '='
T_WHITESPACE
370 ' '
T_LNUMBER
305 '2'
    ';'
T_WHITESPACE
370 '\r\n'
T_CLOSE_TAG
369 '?>'


<? 
  $x = <<<EOT
some string    
EOT  $y = 2;
?>

Actual result:
--------------
T_INLINE_HTML
311 '\r\n'
T_OPEN_TAG
367 '<?'
T_WHITESPACE
370 ' \r\n  '
T_VARIABLE
309 '$x'
T_WHITESPACE
370 ' '
    '='
T_WHITESPACE
370 ' '
T_START_HEREDOC
371 '<<<EOT\r\n'
T_STRING
307 'some'
T_ENCAPSED_AND_WHITESPACE
314 ' '
T_STRING
307 'string'
T_ENCAPSED_AND_WHITESPACE
314 '    \r\n'
T_END_HEREDOC
372 'EOT'         // missing '\r\n' 
T_WHITESPACE
370 '  '
T_VARIABLE
309 '$y'
T_WHITESPACE
370 ' '
    '='
T_WHITESPACE
370 ' '
T_LNUMBER
305 '2'
    ';'
T_WHITESPACE
370 '\r\n'
T_CLOSE_TAG
369 '?>'


<? 
  $x = <<<EOT
some string    
EOT  $y = 2;
?>

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports

 [2005-12-13 10:54 UTC] tomas_matousek at hotmail dot com

Correction:
Expected result of highlighting:
$code = '
<? 
  $x = <<<EOT
some string    
EOT
  $y = 2;
?>'

Actual result:
$code = '
<? 
  $x = <<<EOT
some string    
EOT  $y
 = 2;
?>'

 [2005-12-13 11:31 UTC] tomas_matousek at hotmail dot com

The same result with latest snapshot.
The problem is probably here (file zend_language_scanner.l, around line 1750):

		if (unput_semicolon) {
			unput(';');
		}

I would suggest to replace unputting with 
  yyless(label_len);
which would preserve both ';' and {NEWLINE}.

 [2005-12-13 13:34 UTC] tony2001@php.net

Ilia, please check it out.
The proposed patch looks good for me.

 [2005-12-13 22:45 UTC] iliaa@php.net

This bug has been fixed in CVS.

Snapshots of the sources are packaged every three hours; this change
will be in the next snapshot. You can grab the snapshot at
http://snaps.php.net/.
 
Thank you for the report, and for helping us make PHP better.