Quick Links

File table-based search is lightning fast, so why isn't it included in every major search tool? Read on as we investigate.

Today’s Question & Answer session comes to us courtesy of SuperUser—a subdivision of Stack Exchange, a community-driven grouping of Q&A web sites.

The Question

SuperUser reader Dan Dascalescu is curious why all search isn't table-based:

I've just discovered UltraSearch and was blown away by its file and folder search speed. It's instantaneous. And doesn't use any indexing service. It simply uses the NTFS Master File Table, which already stores all the filenames on the NTFS partition.

The question is, why isn't this capability way more popular among file managers, and Windows Explorer Search (Win+F) to begin with?

We're pretty confident this is the reaction of every computer user upon first encountering just how blisteringly fast file table-based search is. So why isn't it built into everything?

The Answer

SuperUser contributor Mehrdad explains why low-level search has never caught on:

Because of Security!

That's the real reason. (And the only real reason, in my opinion -- it's not that hard to make a reader for major file systems, although it's by no means easy; making a writer is the real challenge.)

A program like this bypasses the entire (file) system's security infrastructure, so only an administrator (or someone else who has "Manage Volume" privileges) can actually run it.

So obviously, it wouldn't work in many scenarios -- and I don't think Microsoft (or any other big company) would ever consider making a product like this and then encouraging users to run as administrators, because of the security ramifications.

It would be theoretically possible to make a system which runs in the background and filters out secured data, but in practice it would be a lot of work to get correct and without security holes for production.

By the way I haven't used UltraSearch, but I'd written a very similar program myself a few years ago which I open-sourced just last month! Check it out if you're interested. :)


Have something to add to the explanation? Sound off in the the comments. Want to read more answers from other tech-savvy Stack Exchange users? Check out the full discussion thread here.