will not work with libdb4.8

Bug #51086	will not work with libdb4.8
Submitted:	2010-02-18 23:41 UTC	Modified:	2010-03-08 20:40 UTC
From:	seanius at debian dot org	Assigned:	sixd (profile)
Status:	Closed	Package:	DBM/DBA related
PHP Version:	5.3, 6 (2010-02-19)	OS:	*
Private report:	No	CVE-ID:	None

[2010-02-18 23:41 UTC] seanius at debian dot org

Description:
------------
while the configure changes needed to get php to build against libdb4.8 are pretty trivial, there's unfortunately a significant behavior change in how libdb's db->open() reacts when called with DB_CREATE|DB_TRUNCATE on an existing zero-sized file.  this also breaks other apps, such as sendmail or nvi, which use similar "locking hacks" (i'm guessing that's what's going on here) on db files.

it's possible that this might be fixed in a later release of libdb4.8 in which case it might be good to refuse building against earlier versions of 4.8 when the fix does come out.  in the meantime i'm wondering about the options for someone who can't back out to 4.7 :/

for reference:

similar bug in sendmail: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=565242

similar bug in nvi (gentoo dev "I'm not sure if this is a bug or a feature"): http://bugs.gentoo.org/286352

Reproduce code:
---------------
run ext/dba/tests/bug36436.phpt, for example, or probably any other db4 related code after building against 4.8.



Expected result:
----------------
PASS

Actual result:
--------------
FAIL, with error messages like:

PHP Notice:  dba_popen(): fop_read_meta: /home/sean/debian/php/ext/dba/tests/test0.dbm: unexpected file type or format in /home/sean/debian/php/ext/dba/tests/bug36436.php on line 6

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports

[2010-02-18 23:46 UTC] seanius at debian dot org

a little more info:  afaict the file is being created with zero size via some stream functions in dba.c.  when db->open() is called with DB_CREATE|DB_TRUNCATE libdb4.8 tries to read some metadata from the existing file and fails horribly.  i tried playing around with the flags (passing only DB_TRUNCATE does not work, passing only DB_CREATE seems to work but still produces the error message).

[2010-02-19 08:43 UTC] seanius at debian dot org

oops, looks like there was a little race there.  yes i can see the behavior in the snapshot too:

rangda[/home/sean/Download/php5.3-201002160730] ./run-tests.php -p ./sapi/cli/php --show-diff ext/dba/tests/bug36436.phpt

=====================================================================
PHP         : ./sapi/cli/php 
PHP_SAPI    : cli
PHP_VERSION : 5.3.3-dev
ZEND_VERSION: 2.3.0
PHP_OS      : Linux - Linux rangda 2.6.32-trunk-amd64 #1 SMP Sun Jan 10 22:40:40 UTC 2010 x86_64
INI actual  : /home/sean/Download/php5.3-201002160730
More .INIs  :  
CWD         : /home/sean/Download/php5.3-201002160730
Extra dirs  : 
VALGRIND    : Not used
=====================================================================
Running selected tests.
TEST 1/1 [ext/dba/tests/bug36436.phpt]
========DIFF========
001+ Notice: dba_popen(): fop_read_meta: /home/sean/Download/php5.3-201002160730/ext/dba/tests/test0.dbm: unexpected file type or format in /home/sean/Download/php5.3-201002160730/ext/dba/tests/bug36436.php on line 6
========DONE========
FAIL Bug #36436 (DBA problem with Berkeley DB4) [ext/dba/tests/bug36436.phpt] 
=====================================================================
Number of tests :    1                 1
Tests skipped   :    0 (  0.0%) --------
Tests warned    :    0 (  0.0%) (  0.0%)
Tests failed    :    1 (100.0%) (100.0%)
Expected fail   :    0 (  0.0%) (  0.0%)
Tests passed    :    0 (  0.0%) (  0.0%)
---------------------------------------------------------------------
Time taken      :    0 seconds
=====================================================================

=====================================================================
FAILED TEST SUMMARY
---------------------------------------------------------------------
Bug #36436 (DBA problem with Berkeley DB4) [ext/dba/tests/bug36436.phpt]
=====================================================================

[2010-02-19 08:45 UTC] jani@php.net

-Summary: will not build/work with libdb4.8
+Summary: will not work with libdb4.8
-Operating System: Debian (and others)
+Operating System: *
-PHP Version: 5.3.1
+PHP Version: 5.3, 6 (2010-02-19)

At least it builds now. :)

[2010-02-19 09:00 UTC] seanius at debian dot org

-Summary: will not work with libdb4.8
+Summary: will not build/work with libdb4.8
-Operating System: *
+Operating System: Debian (and others)
-PHP Version: 5.3, 6 (2010-02-19)
+PHP Version: 5.3.1

sorry for all the noise, when trying the latest snapshot it apparently unpacked on top of a previous snapshot in which i had been investigating the issue.  the tests fail in the new snapshot with more than just the NOTICE error (i had been playing with the open flags as mentioned previously).  the actual failure  diff looks like:

TEST 1/25 [ext/dba/tests/bug36436.phpt]
========DIFF========
001+ Notice: dba_popen(): fop_read_meta: /home/sean/Download/php5.3-201002190730/ext/dba/tests/test0.dbm: unexpected file type or format in /home/sean/Download/php5.3-201002190730/ext/dba/tests/bug36436.php on line 6
002+ 
003+ Notice: dba_popen(): /home/sean/Download/php5.3-201002190730/ext/dba/tests/test0.dbm: Invalid type 5 specified in /home/sean/Download/php5.3-201002190730/ext/dba/tests/bug36436.php on line 6
004+ 
005+ Warning: dba_popen(/home/sean/Download/php5.3-201002190730/ext/dba/tests/test0.dbm,c): Driver initialization failed for handler: db4: Invalid argument in /home/sean/Download/php5.3-201002190730/ext/dba/tests/bug36436.php on line 6
001- resource(%d) of type (dba persistent)
002- string(3) "XYZ"
003- string(1) "X"
004- string(1) "Y"
005- ===DONE===

[2010-02-19 09:05 UTC] seanius at debian dot org

-Summary: will not build/work with libdb4.8
+Summary: will not work with libdb4.8
-Operating System: Debian (and others)
+Operating System: *
-PHP Version: 5.3.1
+PHP Version: 5.3, 6 (2010-02-19

heh, seems we're stepping on each other's toes now.  i'll set the stuff back that i just clobbered, and promise to be quiet for a few hours :)

actually it won't build correctly against db4.8.  i had to modify the snapshot to link against db4.8, as otherwise you see http://bugs.php.net/bug.php?id=51062 , though apparently that's a bogus issue, hrm... :)

[2010-03-02 17:12 UTC] sixd@php.net

The Berkeley DB developers are reviewing this.

[2010-03-05 07:54 UTC] sixd@php.net

-Status: Assigned +Status: Closed

[2010-03-05 07:54 UTC] sixd@php.net

The next patchset of Berkeley DB 4.8 will possibly have the root cause
fixed and the undefined behavior that DBA was depending on reverted.
In the meantime I've merged a fix and a workaround to PHP 5.2.14-dev,
PHP 5.2.3-dev and PHP 6.0.

Note: now when using Berkely DB 4.8 prior or equal to 4.8.26, the
workaround causes a message regarding meta data to be suppressed when
opening the database.  This causes a diff in a few cases where that
message was previously displayed in DB 4.7, but prevents the message
incorrectly displaying in all other tests.

[2010-03-06 11:46 UTC] seanius at debian dot org

Just a thought: what about leaving this open until oracle releases a new libdb, and then committing a second patch that refuses to accept db4.8 < the fixed version via config.m4?  either way, thanks for looking at this.

[2010-03-08 20:40 UTC] sixd@php.net

That seems restrictive, given that normal functionality should be fine with BDB 
<= 4.8.26.