Just to start, I’d like to wish everyone a happy new year.

Here’s a little background. I’ve been trying to design an asset management system for data hoarders. I’m not looking for software suggestions as I find they all tend to be lacking in two main areas that I’m about to describe. This will be a bit long as I try to catalog my requirements and thought process in detail.

Here’s my hypothetical scenario:

• I have a folder that contains n > 100 images. Each image is considered “loose”, in that they have no relation to one another. Unlike say, each page of scanned comic book which need to be grouped and ordered together in a single entity for each image to make sense.
• I want to sort this folder and I have two ways in which to choose; By graphical content (e.g. cars, nature, animals etc) or by function/utility (e.g. memes, wallpapers, reaction images, work). And then afterwards, moving each image to the folder most relevant to it.
• These two methods are mutually exclusive, in that if I choose to organize in one way, I cannot do so in another (technically untrue, but that method involves file duplication so I’m keen to avoid that where possible).
• Secondly, no matter which method I choose to employ, the Information Loss problem occurs as each image’s candidacy for a folder is determined by only a single attribute. For example, If I have an image of a dog jumping to catch a ball in a field. The image falls under the domain of “Animals”, but “jump” and “ball” and “grass” and “park” and so on are lost.
• Thirdly, there is the matter of what to do when an image fits the critera of two folders. Say I have an image that is the painting of a waterfall. Would it fall under “Nature” or “Paintings”?

Luckily, these problems have already been solved. Unfortunately my desired method of implementation has not.

The way I see a lot of software designers tackle this has been to create their program on top of the file system, utilize an in-program tagging system and keeping a database of the tag mappings and then abstracting the duties of sorting to their program. Most “photo managers” and specialized digital management software do this.

Listening to some of the feedback on the internet, this is an imperfect arrangement due to the fact that the semblance of order created by the software is merely an illusion (well, even more of an illusion considering folders themselves are just an abstraction), as change is not propagated down to the file system level meaning if we were to be de-coupled from the application in question, our changes will be lost.

Nobody wants to spend hundreds of hours organizing then losing it in a corrupted database or finding out its pointless because something else better was just developed and theres no way to export your work into this new program or a hundred other ways things can go wrong.

My desired method to go about doing this is to ditch the abstraction. Work directly in the file explorer, as this is something I believe most would find convenient and intuitive. However I have two main roadblocks.

I would like to replicate in-program tags using metadata tags. While some file types do support a “tag” attribute like .jpgs, many like .pngs and .gifs do not. How do I get started with custom metadata programming? I say “custom” as I would like to stay clear of window’s native metadata support as those have a nasty habit of being wiped by other programs or non-supporting file systems, and I would like something more robust.

File organization can be achieved through what I call “context-sensitive-smart-folders” or CSSFs (as regular smart folders already exist). CSSFs differ from smart folders who only remember and output past queries, by being able to toggle between multiple “views” of a folder. In my earlier example of choosing to sort between graphical content and file utility, a CSSF would be able to accomodate both approaches by changing the structure of the GUI output depending on the desired context at the click of a button. Of course, provided they have already been sorted into their respective directories by the user prior. There’ll be other features such as tag searching and drag’n’drop but they’re not really the focus.

Problem here is, I’ve done enough basic research on NTFS’ MFT to know this cant be done through actual folders as theyre simply table entries. Hence CSSFs, like smart folders will be folders in name only. Some preliminary research leads me to Shell Namespace Extensions. Im not familiar at all with WINAPI, so if there’s an easier way I’d like to hear it.

In summary:

file metadata – how to add custom metadata, suggest resources? what should I look out for?

file organization/custom folder programming – do I use shell namespace extensions? Are there alternatives? Are there any good resources out there? What should I look out for?