Metadata in OS X: Wrap Up
I think we’ve covered a lot of ground in the past 4 weeks, talking about Metadata in OS X. While it’s clearly a large step in a new direction, attacking that new way of thinking can really pay dividends as you hone that system to your own needs. In this wrap-up post about Metadata, I’ll share some of the ideas I’m trying to work out, which may even take tagging to the next level of usefulness.
But before we dig into my dark and twisted mind, I wanted to address a question that has come up a couple times in the past weeks. I’m excited to finally be able to share the details behind the robot icon in my menubar. As some of you have surely guessed, it’s from BigRobotSoftware, who have in the past offered Meta, which is a somewhat enhanced Spotlight application. Well as of today you can check out their newest offering, TagBot. I’ll cover it in a little more detail soon, but here’s the quick and dirty:
Drag and drop files onto a window of tags (automatically prepended with the &) for simple tagging of single or multiple files. Double click the tags for an instant Spotlight search window. Or better still, add tags to files (or search for similarly tagged files) via a right click contextual menu item. It’s no frills and works really well. It’s a handy little application on its own, but even moreso if you’re not a Quicksilver user, but want to easily tag your files. The guys at BRS have let me test drive it for a while, and I highly recommend giving it a look.
The Future of Tagging
So you’re getting into the habit of tagging your files and searching for them via Spotlight or Quicksilver. Where do we take it from here?
I’m optimistic that Leopard will heavily integrate tagging into the operating system interfaces. I have a hard time believing that the smart people at Apple would continue to overlook this most awesome feature. Should they be releasing added functionality, I’d love to see the ability for programmers to hook into the metadata functions of the OS for more universal use of the tags. For instance, it would be killer to have non-proprietary tagging available within iView Media Pro. (I realize it’s Microsoft owned now, and not likely to happen, but you get the idea.) Hopefully we’ll know more about Leopard soon!
But how can we take tagging to the next level now? I’ve been tinkering with an idea for a while now, that I think would really make the use of tagging files absolutely blow up. Picture this:
As you create files on your hard drive, tag them, and edit them, a Folder Action (or something of the like) is intently ‘watching’ in the background. A rule (based probably on some fancy Apple Scripting) would evaluate to determine if there were 5 or more files with a matching tag, that had been opened or modified in the past week. Those files would then cause a Smart Folder to be generated on the Desktop, as they’re clearly files that are important in the here and now. Should the date in which those files were last touched slip out of the last 5 day range, the Smart Folder disappears from the Desktop, assuming you’re no longer in need of them.
Basically, if I’m putting in the time to tag my files appropriately, I’d like the OS to do the rest of the thinking for me. Show me what’s relevant to me now and leave the rest of the stuff where it’s at for whenever I decide to go ‘digging’ for it. I think that with some tricky Apple Script-fu, this could be a reality. However I’m not currently the one to produce it.
Possible Solutions
But as I’ve tried different approaches to making a system like this a reality, I’ve exhausted Automator’s limits (or rather, my knowledge of Automator’s abilities). But I think recently I’ve come up with a decent alternative. It’s not the full system I’ve imagined, but a good compromise, should it pan out as I hope.
Hazel sort of acts as a Folder Action, but gives a lot more flexibility to the user (especially if they’re Apple Script challenged, such as I am). There is still a little fore-though required, which means it’s not as automatic and dynamic as I’d like, but it may suffice for the time being… But the setup - for use with Hazel - would go something like so:
Create a Smart Folder looking for the Spotlight Comment &now and save it to the Desktop. Then create a rule for each tag that you plan to use frequently. Each of those rules would additionally look for an opened/modified date of the past 5 days.
This would effectively put your recently important files of specific tags right at your fingertips on the desktop. Of course you’d still need to remove the &now tag from the file once it’s no longer immediately important, but it’s a step in the right direction. I’m still playing with different rules in Hazel to see what may work the best. But the ability to attach Apple and Shell Scripts to your Hazel rules is something that I really like, and hope to leverage moving forward along these lines.
If you’ve got some snazzy Apple Script skills - or other ideas you think may be useful to this vision - I’d love to hear your thoughts!
What Else?
In The Apple Blog Community, one super smart user had the idea of using GeekTool to keep track of his tags generated from within Quicksilver. The tags are stored in an XML file which can have a shell script run against it, and then displayed on your desktop via GeekTool. This way you can quickly and easily keep track of your tags, should the listing get to be more than you can handle without assistance.
I’m sure there are more excellent ideas out there for leveraging Metadata and the wonderful tools available to us on the OS X platform (both baked-in and shareware), and would love to hear any other wild and crazy thoughts you may have on the subject. Please share these, and any other comments below!
And in the meantime, just try to think in a more meta state of mind. Don’t let the history of folder hierarchy keep you confined to such a limited approach to storing your files - there’s much more flexibility at your finger tips than you may realize. Force yourself into the habit of tagging and searching on those tags. Couple this new approach with regular foldering techniques and you may soon find an efficient equilibrium that should make you more productive than ever before.
If you enjoyed this series of articles, let us know! It’s been a lot of work putting them together, but also a lot of fun. If there are specific topics you’d like to see covered in this same detail over a period of time, share with us and we’ll do our best to bring it to you.
Trackbacks/Pingbacks (Trackback URL)
Comments RSSComments
#1 Eric Hamilton says:I’ve been eagerly anticipating TagBot and Pete from BigRobotSoftware was kind enough to alert me to its availibility early this morning. I’ve been playing around with it for the better part of the morning and it definitely blows the roof off of other tagging methods. The ability to look at the pool of your tags definitely keeps me more organized and does away with problem duplicate tags such as “apps” v “app” etc. The interface is quite elegant and the ability to drag & drop files for tagging is a nice option to have. I’ve been using Quicksilver to tag my files for several months and TagBot will definitely replace QS in that regard. One last note; TagBot prepends tags with the ampersand in the same fashion as I had set up in QS which makes the transition seamless.
#2 Chris Pickett says:I kind of posted this in the screencast thread, but you may not have seen it, or had an answer…
Basically how do you deal with tags that have a lot of files associated, say 100+, in the folder hierarchy world you can make subfolders, but I haven’t found a way to make “sub-tags”, or I guess more to the point use multiple tags to find a file. Is there anyway to do this, or even a related tag feature?
#3 Chris says:Maybe I’ve missed it in an earlier screencast, but is there a way to bulk delete all Spotlight comments from a group of items? I’d like to rework the way I use metadata and want to start from a clean slate. Google turns up nothing and I’ve tried creating a workflow in Automator using the “Add Spotlight Comments” action and replacing rather than appending a single space, but that doesn’t work either.
#4 Ryan Perry says:Chris:
I’m not sure that you’re asking your question with regard to TagBot, or with regard to Quicksilver’s tagging functionality, but I’ll just throw this out there either way…
I think what you want to do is to apply multiple tags to your files. So if, say, you were organizing a bunch of photos, you might apply a tag called “vacation” to all your photos that came from any vacation. Then you’d apply a tag called “Paris” to any photos from Paris, regardless of whether they’re from your vacation or not.
Then, when you want to find your Paris vacation photos, you’d select the “Paris” tag and the “vacation” tag in TagBot, then hold shift while double-clicking them. When you do this, TagBot will find for you all the photos you’ve tagged with “Paris” and “vacation”.
The effect is similar to if you’d had a folder called “vacation”, and it contained a sub-folder called “Paris”, and you kept your Paris vacation photos in there. But the thing is, tagging is a much more flexible way to organize, because what if you also want to organize based on the year in which your photos were taken? Would you keep your “Paris” folder inside your “2006″ folder? What if you wen to Paris in 2006 and 2004? Sure, you can just keep two folders called “Paris”, one in each year, but that can get to be a hassle if you want to find all your Paris photos, regardless of the year.
If you’re using tagging to organize, you just tag your 2006 photos with the “2006″ tag, and the 2004 photos with the “2004″ tag. Then if you want to see your Paris photos from 2004, you ask TagBot to show you files with the “2004″ and “Paris” tags. And if you want to see all your Paris photos, you just ask TagBot to show you all files with the “Paris” tag.
To summarize, I think your best bet is to apply multiple tags to your files in cases when you’d be putting your files into sub-folders if you were organizing by folder. I’m sure this system would work with Quicksilver as well as it would with TagBot, but TagBot makes it pretty easy to see all your tags and select the set of tags you want to access.
Hope this helps!
#5 Chris Pickett says:I guess it was more of an “in general” question. I understood multiple tags, but other than creating a smart folder and specifying multiple comments, I couldn’t come up with a quick way of nailing it down farther. I’ll try TagBot out, that sounds like exactly what I was looking for. Thanks!
#6 Ryan Perry says:Hi again Chris,
If you have a copy of Meta, you can use it to edit Spotlight comments for multiple files at the same time. Just select the files you want to clear Spotlight comments from in the Meta search results pane, then right-click and select “Edit Spotlight Comments”. Once the Spotlight Comments window pops up, you can clear the comment field and click OK.
#7 Jono says:I’ve really enjoyed all your metadata posts. They’ve been really informative & have made me get into tagging in a big way.
You ideas sound great. I hope someone with more brains than me can get it up & running.
TagBot looks good, kinda similar to Punakea. Looks like both have their plus & minus points points:
I really like the context menu plugin for TagBot but it looks like it should have an ‘Add New Tag’ option.
I like the fact that it uses the ‘&’ prefix rather than Punakea using it’s own tagging characters method (although you’ll be able to choose in the future).
I like the way in Punakea that you can drag a tag off the tagging window & it’ll create a smart folder on the desktop. I think that feature would be welcome in TagBot (I found a sort of way of doing with this with the smart searches brought up with Quicksilver. If you click on the smart folder icon in the title bar (next to the name of the tag) & drag it onto the desktop it creates a smart folder for the search).
Would be nice if you you dock the tag palette to the edge of the screen & show/autohide it when needed (a la Punakea).
Thanks again for the series!
#8 Ryan Perry says:Hi Jono,
It’s not very obvious, but in TagBot you can save a tag or set of tags as a Smart Folder by right-clicking on the tag(s), then selecting Save As Smart Folder. But the other stuff you suggest sounds interesting.
#9 NotVeryPC says:“In The Apple Blog Community, one super smart user had the idea of using GeekTool to keep track of his tags generated from within Quicksilver.”
I wouldn’t go that far, but thanks.
#10 Charlie Sorrel says:Wowee Zowee!
This is sweetness indeed. Especially the ‘hidden’ feature you pointed out in the comments up there for ‘AND’ searching by shift clicking.
OK. I have been using the ‘@’ to perpend, and my QS tags all use this. I want to go over the ampersand now. Any idea on how to do this? I could do it manually without too much hardship, I guess, but it’ll take an age as I have 124 tags set in QS.
Still and all, TagBot looks like a keeper. I’ll give it a few days but I can see 20 bucks winging it’s way to you pretty soon.
#11 Charlie Sorrel says:Also, don’t all you good people forget that Dragthing will read smart folders, so you can stick a bunch in your Dragthing dock and right click to drill down into them.
For instance, I have a Smart folder for “today”. Anything I opened already today is in there, and that one alone is a huge timesaver.
http://www.dragthing.com
#12 Galley says:I hope Leopard includes an option to prompt for spotlight comments whenever you save a file.
#13 Jono says:Thanks Ryan
One thing I noticed with TagBot is that in Mail the keyboard shortcut far adding attachments is CTRL, Shift +A (which I use all the time). So at the moment it’s not playing nicely with TagBot. Maybe we could have the option to change the hot key in the future?
Charlie you could retag your files like this:
Bring up the tag you want to change in Quicksilver & select ‘Show Files Matching Tag’ which will create a Spotlight Finder search window. (Say for example they are called @project1).
Create a tag in TagBot called project1 & drag all the files (tagged at the moment with the @ prefix) from the Spotlight Finder search window onto the TagBot ‘project1′ tag. They will now be tagged with the &project1 tag (as well as the @project1 tag).
Then select all the files again in the Spotlight Finder search window & bring them up in Quicksilver (I have a trigger to do this but if you don’t you could go) Finder Selection or Current Selection in the 1st pane, Tab, Remove Tags in the 2nd pane, Tab project1 in the 3rd pane. This will remove the @project1 tags & leave the &project1 tags intact.
Then change the tags prefix from @ to & in Quicksilver just to be on the safe side for the future.
#14 Charlie Sorrel says:Jono:
Thanks. Good idea. I probably have a whole bunch of junk tags in there anyway
so this’ll help with trimming them out.
Ryan:
I notice that email and address book entries don’t show up in the Smart folders. This is an issue with OS X: They show up in a Spotlight search but not in Smart Folders unless you manually edit the Smart Folder in a text editor.
Would it be possible to remove these lines form the query TagBot sends to the Finder:
&& (kMDItemContentType != com.apple.mail.emlx)
&& (kMDItemContentType != public.vcard)
(as detailed on MacOSXHints here)
That would be most awesome, as then TagBot will find my MailTagged mails.
I notice that your other app, Meta, finds these (mail and contacts) but the saved Meta searches won’t work with QS like regular Smart Folders will.
Thanks!
#15 Chris Pickett says:Ok, I’ve been using TagBot for a little while now (actually just ponyed up the $20 for it and for meta). It’s really great and has helped a lot, the only thing I get annoyed by is that I can’t use the keyboard to either arrow down through the list, or type a partial tag to go to matching tags. Anyway to get that in a future release, or does it do it and I’m just a moron?
Also with meta, it would be nice if it interfaced with quicksilver, so that you could put qs into text mode and then run the text search in meta.
#16 Chris Pickett says:An addition to my last post… TagBot will let you arrow through them, but only after you click the window. I suppose the real problem is that it doesn’t take keyboard focus automatically.
#17 Ryan Perry says:Ok, I’m going to try to respond to a bunch of comments here…
Jono:
Sorry for the hotkey collision. We tried to find a hotkey that would be fairly collision-free, but I guess we’ve found a collision for this one. Pete and I are probably going to try to work out a good way to change the hotkey for the next release. Thanks for pointing this out.
Charlie:
I’m confused by the fact that TagBot isn’t finding emails and AddressBook entries. I’m pretty sure we’re not adding those lines you pointed out to our queries. Send me an email at ryfar-at-bigrobotsoftware-dot-com so we can discuss this further, if you don’t mind. I’d like to figure out what’s going on.
Also, I’m not sure how interoperable TagBot and MailTags are right now, but I’m going to look into it.
Chris:
I’m glad you liked TagBot and Meta enough to buy them; thanks!
Originally, Pete and I didn’t want Meta to take focus when the window was brought forward, because we wanted it to act less like a separate application and more like just a regular part of the system. But you raise a valid point about arrowing through tags. We’re going to look into the ramifications of making it take focus when being un-hidden. Thanks for bringing that up.
Cheers,
Ryan
#18 Alvin Law says:I’m been so tired of having a messy desktop. And it’s not even about that, it’s just i have a difficult time organizing all my different files. I think metadata may be my solution, i’ll keep reading your articles. It’s been great to read and when I see a solution, i will implement. Thanks for writing and sharing, i’ll be back!
#19 Jono says:No problem Ryan. I can use the icon on the toolbar in Mail to add attachments for now
For anyone else tagging files with Quicksilver. I wasn’t loving the standard tag icon for the plugin, so I created some new ones. You can view, download them & get the instructions on how to change the default one for the new ones here http://docs.blacktree.com/quicksilver/creations/replacement_icons (bottom of the page)
#20 Blastdoor says:Tagbot looks interesting — but does it allow you to organize a large number of tags? It looks like they all get dumped in the same palette — is there any way to have sub-palettes of tags, or some other way or organizing the tags?
#21 Jono says:Yea, I was thinking about that. I have quite a lot of tags. So maybe organising them into sub menus could work? Could possibly have groups & when you drag a file onto the group or small triangle on the left of the group it drops down to reveal the tags in the group? Then once youdropped the file onto the tag it springs back up to hide them again. Or maybe a favourites &/or recent tags part could work?
#22 Yaqui says:nick, thank you very much for this series of articles. i’ve found them very useful and have followed up on all of them. i’ve been using this ’system’ for a month and am quite happy with it. i had done the same idea of making a smart folder in my desktop and querying on &inuse tag. it’s very helpful when i’m working in a campaign (i’m an advertising creative). well, just wanted to let you know, thanks and keep those screencasts coming. best regards.
#23 Ryan Perry says:Blastdoor and Jono:
We’ve had a few other people ask us about organizing and / or categorizing tags as well. It was an issue we’ve been aware of, but for version 1.0 we wanted to release it as you see it now, and to gauge user demand for the ability to organize tags.
So thanks for adding to that conversation. Pete and I still need to discuss the feature some more, and figure out if it’s a good fit with TagBot, and if so, how it should work. Feel free to email us (support_bot-at-bigrobotsoftware-dot-com) if you’d like to discuss it some more.
#24 Brandon Payne says:When the geektool instructions say just call the script from geektool, how do I do that?
#25 NotVeryPC says:Brandon
You need to copy the script into “Script Editor” and save it as “GetTags.scpt”
In GeekTool create a new entry with the command:
osascript /Users/username/path/to/script/GetTags.scpt
Hope this helps.
#26 Old Shatterhand says:Neither Quicksilver, nor Tagbot, nor Spotlight properly handles accentmarks, we Europeans are so used to. The apps don’t distinguish between e.g. e and é (e with ‘accent aigu’).
I can’t remove them with QS nor Tagbot. Now I am stuck with a few tags that have accentmarks in them. No problem to add another tag without it, but I would like to find an easy way to delete the accented ones.
My advise for the moment: avoid accentmarks in your tags!
#27 Ryan Perry says:Old Shatterhand:
The accent mark issue was recently brought to our attention, and we’ve been working on fixing it for an upcoming release of TagBot.
In the meantime, you should be able to delete the accented tag from any files you’ve applied it to by right-clicking on the file in the Finder, selecting Get Info, and deleting the tag text from the Spotlight Comments field in the Get Info window.
Sorry for the inconvenience; we’ll have a free upgrade with a fix for this issue released pretty soon.
#28 Ryan Perry says:Just thought I’d mention, for posterity, that we’ve just released TagBot 1.01, which addresses some of the Unicode / accented character issues, as well as the hotkey collision issue. TagBot should alert you to the update automatically within a week or so, or you can download the new installer from http://www.bigrobotsoftware.com.
Thanks for everyone’s input! For further questions or support, please email us at support_bot-at-bigrobotsoftware-dot-com.
And thanks to Nick for mentioning TagBot! He’s been a great source of interest and feedback.
#29 Dave M. says:I came across a huge problem with using Spotlight Comments for tagging. I came across this when I was updating software that I had recently tagged. I lost those tags.
An Application on the Mac is really just a folder named {appname}.app right? Well, what happens when you copy a newer version of that app from a DMG file? Doesn’t it replace the folder that contains the tags with the one in the DMG file that doesn’t have tags?
If I were to go into the contents of the app in the DMG and copy all the files to the app on my HD, then I would expect the tags to stay since I’m not touching the folder that represents the application.
Am I missing something here? I’m still a bit new with Mac and OS X so its certainly possible.
#30 ip says:Hi Nick,
I have written XTagFS - a macfuse filesystem that allows you to view tags (i.e. spotlight comments) and tagged files as a file-system. In XTagFS, a tag becomes a folder which contains (alias/links to) files tagged with that tag. Files tagged with multiple tags can be accessed in any order: e.g. file tagged with tag1 and tag2 can be found at path /tag1/tag2 or tag2/tag1. Tagging/Untagging files and managing tags using file-system operations (mv, rm, etc) is planned for the next release.
You can get XTagFS from here: http://code.google.com/p/xtagfs/
Please give it a shot - I would love to have some feedback/comments!
Thanks
ip
#31 Ryan Perry says:Hi Dave M,
I’m not an Apple engineer, but to the best of my knowledge, the Finder associates Spotlight Comments with the actual file, and not with the location of the file. Thus, when you move a file, its Spotlight Comments go with it. And that’s probably a good thing.
When you replace an existing file with another file, you are really deleting the original file and putting the new file in the same location with the same name. So because the Spotlight Comments are associated with the file, and not the location, and because you’re deleting the original and putting another in its place, the Spotlight Comments of the original file to the same place that the original file goes: they get deleted.
That’s why copying a new version of an application over an old version causes the Spotlight Comments you’d associated with the old version to go away. So yeah, that’s something to watch out for.
Hope this helps.
#32 Dave M. says:Ryan,
I know why my spotlight comments were being removed/lost. I was just pointing out that using spotlight comments for tagging really doesn’t work if they get lost when an application is updated.
I love the idea of tagging apps, I just wish there was a more permanent way of doing it.
#33 Derick says:Will Spotlight comments travel if the file is copied from one Mac to another? I have a desktop and laptop that I keep in sync with ChronoSync but I don’t want to invest a lot of time tagging if the tags won’t sync along with the files.
A related question: for PDF files, would it not be better to use the PDF keyword field (accessible via Preview) as the basis for tagging PDFs since these are embedded in the file and thus guaranteed to travel with the file across machines, operating systems etc.?
#34 Dave M. says:Derick, yes, the Spotlight comments will be copied with the file. The Spotlight comments are embedded with the file you put them in. So they will travel with the file when being copied like PDF files and JPEG EXIF data.
The problem with this tagging method is when you update an application.
#35 Michael Gregory says:I’ve discovered a Tag’s not traveling problem with PathFinder.
I just purchased TagBot as well and I thought it would tandem well with PF (Tabs!)for tagging and then distributing unsorted files to folders… but PF appears to be stripping the Tags from the Spotlight Comments when the file is dragged and dropped across windows. I realized after sorting about 30 files. Testing with Finder the comments remain intact.
#36 Alain L From France. says:Ryan, i really like your apps, but i see there, Tags can disappear… in case of update, right ? Should’nt you mention it ? I’m not able to uninstall TagBot, wich make my contextual Menu break and crash the system… What did i wrong ? It’s ok for French translated 1.1 version soon.
#37 Adrian Smith says:I’m really keen to start using Spotlight Comments to tag my files in the finder but I’m concerned by the comments above about loosing tags and by Rob Griffiths comments on the fragility of Spotlight Comments (see http://www.macworld.com/weblogs/macosxhints/2005/11/spotatsix/index.php> (basically implies doing almost anything to move files around (other than using the finder) will kill the tags.
Similarly I believe that Scott Morrison used Spotlight comments for MailTags originally but abandoned them quickly when he realised how fragile there are.
Any comments from the experts here?
#38 amurgrades says:Clearly. Thanks!
:))
Leave a reply