Optimize PHP function

Question

I have a function that detects all files started by a string and it returns an array filled with the correspondent files, but it is starting to get slow, because I have arround 20000 files in a particular directory. I need to optimize this function, but I just can't see how. This is the function:

function DetectPrefix ($filePath, $prefix)
{

    $dh  = opendir($filePath);
    while (false !== ($filename = readdir($dh)))        
    {   
        $posIni = strpos( $filename, $prefix);
        if ($posIni===0):                           
            $files[] = $filename;                       
        endif;
    }

    if (count($files)>0){                               
        return $files;
    } else {
        return null;
    }

}

What more can I do?

Thanks

How about changing that directory structure? You could create subdirectories based a the first characters of the file names, which would make it even faster to find the relevant files. No need to look into a/a/aa* when you're looking for bb* — sisve
– sisve, Commented Jul 8, 2009 at 14:23
glob may help for a time, but KM is right: in the long run you have to do something about your directory structure. having that many files in one directory can lead to a lot of problems. — stefs
– stefs, Commented Jul 8, 2009 at 15:06
I just can't do nothing about the huge number of file in that directory. Ancient software: must live with it ;) Thank you for all answers! — RSilva
– RSilva, Commented Jul 9, 2009 at 9:47

ceejayoz · Accepted Answer · 2009-07-08 15:36:24Z

11

http://php.net/glob

$files = glob('/file/path/prefix*');

Wikipedia breaks uploads up by the first couple letters of their filenames, so excelfile.xls would go in a directory like /uploads/e/x while textfile.txt would go in /uploads/t/e.

Not only does this reduce the number of files glob (or any other approach) has to sort through, but it avoids the maximum files in a directory issue others have mentioned.

edited Jul 8, 2009 at 15:36

answered Jul 8, 2009 at 14:12

ceejayoz

181k42 gold badges311 silver badges382 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Community · Accepted Answer · 2023-11-17 19:24:28Z

4

You could use scandir() to list the files in the directory, instead of iterating through them one-by-one using readdir(). scandir() returns an array of the files.

However, it'd be better if you could change your file system organization - do you really need to store 20000+ files in a single directory?

edited Nov 17, 2023 at 19:24

CommunityBot

11 silver badge

answered Jul 8, 2009 at 14:14

Peter

6,3821 gold badge34 silver badges30 bronze badges

Comments

KM. · Accepted Answer · 2009-07-08 14:41:28Z

2

As the other answers mention, I'd look at glob(), scandir(), and/or the DirectoryIterator class, there is no need to recreate the wheel.

However watch out! check your operating system, but there may be a limit on the maximum number of files in a single directory. If this is the case and you just keep adding files in the same directory you will have some downtime, and some problems, when you reach the limit. This error will probably appear as a permissions or write failure and not an obvious "you can't write more files in a single directory" message.

answered Jul 8, 2009 at 14:41

KM.

104k34 gold badges183 silver badges214 bronze badges

Comments

raspi · Accepted Answer · 2009-07-08 14:15:17Z

1

I'm not sure but probably DirectoryIterator is a bit faster. Also add caching so that list gets generated only when files are added or deleted.

answered Jul 8, 2009 at 14:15

raspi

6,1423 gold badges36 silver badges52 bronze badges

Comments

Gumbo · Accepted Answer · 2009-07-08 14:16:08Z

0

You just need to compare the first length of prefix characters. So try this:

function DetectPrefix($filePath, $prefix) {
    $dh  = opendir($filePath);
    $len = strlen($prefix);
    $files = array();
    while (false !== ($filename = readdir($dh))) {
        if (substr($filename, 0, $len) === $prefix) {
            $files[] = $filename;
        }
    }
    if (count($files)) {
        return $files;
    } else {
        return null;
    }
}

answered Jul 8, 2009 at 14:16

Gumbo

657k112 gold badges792 silver badges852 bronze badges

1 Comment

H_I Over a year ago

That's true, but IMHO, this is more a matter of I/O, than of in-memory treatment.

Collectives™ on Stack Overflow

Optimize PHP function

5 Answers 5

Comments

Comments

Comments

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

Comments

Comments

Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related