Home Original page

No space left on device

Ron Johnson ron.l.johnson at cox.net
Wed Feb 20 23:32:00 EST 2002 
Creating 100,000 files like a pretty sub-optimal solution to almost
any problem.  Have you considered using a database, or an indexed file?
Maybe 1 file where each chunk of data is appended to the end, and a 
2nd file containing offset pointers into The Really Big File.

If, on the other hand, creating 100,000 small files is the best, or
only, way to do it, then think about moving to linux and using the
Reiser FileSystem.  It does _extremely_ well with small files.  And,
as a side-benefit, it is journaled, like NTFS.

On Wed, 20 Feb 2002 16:23:13 -0700 sag at hydrosphere.com (Sue Giller) wrote:

> Curiouser and curiouser
> 
> I don't think the problem is in the space on the drive.
> 
> I can get the program to run completely with no slowdown if I 
> generate filenames without using the mktemp function
> 	fileName = base + number + extension
> 
> I can also get the program to run completely, with no slowdown, if I 
> generate filenames, BUT don't make any files.
> 
> There seems to be some issue with the combination of using 
> mktemp to get a unique filename (the slowdown is probably due to 
> some sort of checking mktemp does for each name generation) and 
> also creating a file.
> 
> My original code generated a series of temp files using one of three 
> different seeds, but it failed also at the 21800 level.  So changing the 
> seeds doesn't cure this.
> 
> I will try a test where I build a directory tree and see if that fixes 
> things.  Currently, I am inclined to think there is something in 
> mktemp and my computer (ram size or some such) that might be the 
> problem.  I need to try this on a drive not on my main machine  as 
> well.
> 
> Thanks for replies.
> 


-- 
+------------------------------------------------------------+
| Ron Johnson, Jr.        Home: ron.l.johnson at cox.net        |
| Jefferson, LA  USA      http://ronandheather.dhs.org:81    |
|                                                            |
| 484,246 sq mi are needed for 6 billion people to live, 4   !
! persons per lot, in lots that are 60'x150'.                |
! That is ~ California, Texas and Missouri.                  !
! Alternatively, France, Spain and The United Kingdom.       |
+------------------------------------------------------------+
More information about the Python-list mailing list