I got the following to work on Windows 8 / Firefox.
The correct way to register a custom protocol is described in this MSDN article.
Sadly, you cannot apply this directly to the Windows Explorer. If you do, you'll find that the Explorer starts spawning copies until either your memory or your patience runs out, and you'll have to log off to stop it. The reason is that the application handling the protocol is passed the entire link including the protocol specification. I. e., if you have a link
localdir:D:\somefolder,
the resulting call will not be
explorer D:\somefolder,
but
explorer localdir:D:\somefolder.
This is apparently done so that the same application can handle several protocols. But Explorer doesn't recognize that it is meant to handle the request, and instead starts the resolution process anew, which sets the vicious circle in motion.
To deal with this, you call a simple helper app that removes the protocol specification from the argument, and then calls Explorer with the cleaned string.
As an example, I created a bare bones Java class Stripper.
import java.io.IOException;
public class Stripper {
public static void main(String[] args) {
String stripped = args[0].substring("localdir:".length());
try {
Runtime.getRuntime().exec("C:\\Windows\\explorer.exe "+stripped);
}
catch (IOException e) { /* error handling */ }
}
}
The following is a .reg file adding the required keys to the registry. This assumes that Stripper is located in the folde D:\Stripper.
Windows Registry Editor Version 5.00
[HKEY_CLASSES_ROOT\localdir]
@="URL: localdir Protocol"
"URL Protocol"=""
[HKEY_CLASSES_ROOT\localdir\shell]
[HKEY_CLASSES_ROOT\localdir\shell\open]
[HKEY_CLASSES_ROOT\localdir\shell\open\command]
@="cmd /c \"cd /d d:\\Stripper & java Stripper %1\""
The command invokes the command line interpreter, telling it to first change to the directory containing our helper, and then to call the JRE to execute it. It's a bit awkward, but it works.
With a native .exe file, the command key could look like this:
[HKEY_CLASSES_ROOT\localdir\shell\open\command]
@="<somepath>Stripper.exe %1"
This is a quick'n'dirty solution, you'll want to obviously want to add checks and balances and probably more mechanics, but the general approach looks workable.
