A PowerShell script to find the file size and file count of a folder with millions of files?

Get-ChildItem probably reads all entries in the directory and then begins pushing them to the pipeline. In case that Get-ChildItem doesn't work well, try to switch to .NET 4.0 and use EnumerateFiles and EnumeratedDirectories:

function Get-HugeDirStats($directory) {
    function go($dir, $stats)
    {
        foreach ($f in [system.io.Directory]::EnumerateFiles($dir))
        {
            $stats.Count++
            $stats.Size += (New-Object io.FileInfo $f).Length
        }
        foreach ($d in [system.io.directory]::EnumerateDirectories($dir))
        {
            go $d $stats
        }
    }
    $statistics = New-Object PsObject -Property @{Count = 0; Size = [long]0 }
    go $directory $statistics

    $statistics
}

#example
$stats = Get-HugeDirStats c:\windows

Here the most expensive part is the one with New-Object io.FileInfo $f, because EnumerateFiles returns just file names. So if only count of files is enough, you can comment the line.

See Stack Overflow question How can I run PowerShell with the .NET 4 runtime? to learn how to use .NET 4.0.

You may also use plain old methods which are also fast, but read all the files in directory. So it depends on your needs, just try it. Later there is comparison of all the methods.

function Get-HugeDirStats2($directory) {
    function go($dir, $stats)
    {
        foreach ($f in $dir.GetFiles())
        {
            $stats.Count++
            $stats.Size += $f.Length
        }
        foreach ($d in $dir.GetDirectories())
        {
            go $d $stats
        }
    }
    $statistics = New-Object PsObject -Property @{Count = 0; Size = [long]0 }
    go (new-object IO.DirectoryInfo $directory) $statistics

    $statistics
}

Comparison:

Measure-Command { $stats = Get-HugeDirStats c:\windows }
Measure-Command { $stats = Get-HugeDirStats2 c:\windows }
Measure-Command { Get-ChildItem c:\windows -recurse | Measure-Object -property length -sum }
TotalSeconds      : 64,2217378
...

TotalSeconds      : 12,5851008
...

TotalSeconds      : 20,4329362
...

@manojlds: Pipelining is a fundamental concept. But as a concept it has nothing to do with the providers. The file system provider relies on the .NET implementation (.NET 2.0) that has no lazy evaluation capabilities (~ enumerators). Check that yourself.

The following function is quite cool and is fast to calculate the size of a folder, but it doesn't always work (especially when there is a permission problem or a too long folder path).

Function sizeFolder($path) # Return the size in MB.
{
    $objFSO = New-Object -com  Scripting.FileSystemObject
    ("{0:N2}" -f (($objFSO.GetFolder($path).Size) / 1MB))
}

Here's what I usually use inspired by old unix aliases using du and ls, granted a .net method is faster (https://github.com/jdhitsolutions/PSScriptTools/blob/master/docs/Get-FolderSizeInfo.md). I only calculate the top level directories and files.

function com { param([Parameter(ValueFromPipeline=$True)]
  [int64]$number)
  process { '{0:n0}' -f $number } }

function dus($dir=".") { 
  get-childitem -force -directory $dir -erroraction silentlycontinue `
    -attributes !reparsepoint | 
    foreach { $f = $_ 
      get-childitem -force -r $_.FullName  -attributes !reparsepoint -ea 0 | 
        measure-object -property length -sum -erroraction silentlycontinue | 
        select @{Name="Name";Expression={$f}},Sum} | sort Sum |
    select name,@{n='Sum';e={$_.sum | com}}
}

function siz() {
  ls -file -force | select name,length | 
    sort length |
    select name,@{n='Length';e={$_.length | com}}
}

dus

name                   sum
----                   ---
Documents              2,790,100,862
AppData                20,571,019,655
Downloads              25,564,270,200


siz

name                   length
----                   ------
ntuser.dat.LOG2        3,691,520
ntuser.dat.LOG1        3,698,688
NTUSER.DAT             14,680,064