This is the kind of tool that should be baked into the kernel. It's never there when you need it, and when you do need it, it is probably already a full disk and you maybe can't just download it.
Had this happen to me just last week. Had forgotten to change the “to download into” directory in an GOG archive script after migration to a new system. Filled a folder on the system disk instead of my intended ZFS array.
Naturally ncdu wasn’t installed, so it took me a bit to figure out what the heck happened.
It’s interesting and ironic how so many things in proxmox that would normally release space, instead become nonfunctional when the disk is full. Things like deleting old VM disks or cached iso’s will fail via the interface. I think the web console also broke, had to ssh in to manually free space up.
On ZFS it often is old snapshots and it is surprisingly hard to find out how much space will be freed after destroying one.
I found out that 'zfs destroy' has a dry-run option along with one that can show reclaimed space. After this i hacked together a little cli tool [1] that calculates what snapshots need to be deleted to reclaim e.g. 10GB.
It's not really production ready but from time to time it's useful :-)
That looks like a nice tool. It would be helpful if it could optionally run off of a data file. I've used "ncdu" to visualize my restic backups by writing a script that generates the ncdu data format, and more recently I've abused ncdu to visualize process memory usage on macos:
I'm planning on implementing persistence in the near future, likely using some kind of generic embedded database or KV store rather than a serialization format. The primary goal would be to enable resummarization without rescanning directories and consequently, real-time updates from file notifications. However, I don't want to force persistence, since one use case for this tool is answering "why is my laptop hard drive completely full!?!?".
re: memory usage, because of the price of RAM these days, my original plan was for a hierarchical memory usage analyzer, but attribution of shared libraries on Linux isn't straightforward.
"Real" memory usage is unfortunately ill-defined since it could mean "memory freed by killing a process", "memory allocated specifically to the process", "proportional of system memory shared by other processes", etc.
Such a tool would likely need multiple subcommands to reflect how the user's intent, along with an extensive README to explain the subtleties.
To illustrate, if we only show the proportional set size in a hierarchy (e.g. IDE with LSP... I'm looking at you rust-analyzer!), but the user wants to know how much memory could be reclaimed by killing the parent process, two things could happen, as far as I understand. If it's the only hierarchy using a large chunk of shared libraries, the PSS will be pretty close. However, if other apps are using those libraries, very little memory will be freed (mostly what was allocated specifically to the killed app).
On top of that, we also have to consider swap space.
I may revisit this in the future once I have a better understanding of smaps and friends, if RAM prices don't normalize anytime soon.
I've been using Sequoia View regularly since the early 2000s, and it's held up amazingly well considering how much bigger hard drives have gotten in the past 25 years.
I’ve been using ncdu and WinDirStat for years. This looks awesome. One feature I’d love to see is exporting the results as HTML+SVG reports. I maintain a shared server, and it’s often useful to share a report with the team to figure out who’s using up the disk space.
Love this! A few similar GUI apps have been mentioned for Windows. For Mac, I've been using GrandPerspective for probably 15 years. This is one of my favorite class of programs in terms of how uniquely useful they are.
A pity we can't use something like this (and actually take action on the results) on closed systems like iOS (I've just had to factory reset my iphone to rein in a 37GB "system data" mess... even on the newly restored phone that consumes 15GB but at least that's better).
This looks fantastic, reminds me a lot of SpaceSniffer. The focus view or allowing for navigation through chunks is a nice essential inclusion. One desire might be quick actions.
Doing size of squares based on the # of packages a dependency installation causes: Helps I guess users hellbent on having their install minimal figure out what they can afford to remove for as few packages on their system as reasonably possible.
Thanks! I'll need to check out SpaceSniffer next time I'm on Windows.
Can you provide some examples of "quick actions"?
Currently, the visualization is purely based on file sizes in the directory structure. Package management adds some complications beyond the fact that there are at least a dozen popular managers in the wild. For one, package dependencies form a directed graph rather than a hierarchical tree, so credit assignment is vague. Two packages can depend on the same two dependencies. Do we give full credit to both, one or assign partial credit? Would we weight partial credit evenly or by dependent size or some external factor/
Great to see another local-first visualization tool. I'm constantly cleaning up old files just to free up space, so I really appreciate how simple and focused this looks.
I've had on my someday bucket list doing up a web based version of daisy disk as a docker image I can slap on my various servers, but this is probably just as good an approach. I like it.
I had been exploring using an embedded database as an index, but for my current use case, waiting just under a minute to rescan my /nix/store on a weak mini-pc is acceptable.
Also looking to add inotify integration, which would require an index to accurately update the visualization.
Naturally ncdu wasn’t installed, so it took me a bit to figure out what the heck happened.
It’s interesting and ironic how so many things in proxmox that would normally release space, instead become nonfunctional when the disk is full. Things like deleting old VM disks or cached iso’s will fail via the interface. I think the web console also broke, had to ssh in to manually free space up.
I found out that 'zfs destroy' has a dry-run option along with one that can show reclaimed space. After this i hacked together a little cli tool [1] that calculates what snapshots need to be deleted to reclaim e.g. 10GB.
It's not really production ready but from time to time it's useful :-)
1: https://github.com/sandreas/zfs-tool
https://gist.github.com/dunhamsteve/59f5e6b9a4bc69039853674d...
re: memory usage, because of the price of RAM these days, my original plan was for a hierarchical memory usage analyzer, but attribution of shared libraries on Linux isn't straightforward.
"Real" memory usage is unfortunately ill-defined since it could mean "memory freed by killing a process", "memory allocated specifically to the process", "proportional of system memory shared by other processes", etc.
Such a tool would likely need multiple subcommands to reflect how the user's intent, along with an extensive README to explain the subtleties.
To illustrate, if we only show the proportional set size in a hierarchy (e.g. IDE with LSP... I'm looking at you rust-analyzer!), but the user wants to know how much memory could be reclaimed by killing the parent process, two things could happen, as far as I understand. If it's the only hierarchy using a large chunk of shared libraries, the PSS will be pretty close. However, if other apps are using those libraries, very little memory will be freed (mostly what was allocated specifically to the killed app).
On top of that, we also have to consider swap space.
I may revisit this in the future once I have a better understanding of smaps and friends, if RAM prices don't normalize anytime soon.
If possible, being able to “brew install” on a Mac would be killer
A pity we can't use something like this (and actually take action on the results) on closed systems like iOS (I've just had to factory reset my iphone to rein in a 37GB "system data" mess... even on the newly restored phone that consumes 15GB but at least that's better).
Can you provide some examples of "quick actions"?
Currently, the visualization is purely based on file sizes in the directory structure. Package management adds some complications beyond the fact that there are at least a dozen popular managers in the wild. For one, package dependencies form a directed graph rather than a hierarchical tree, so credit assignment is vague. Two packages can depend on the same two dependencies. Do we give full credit to both, one or assign partial credit? Would we weight partial credit evenly or by dependent size or some external factor/
I've had on my someday bucket list doing up a web based version of daisy disk as a docker image I can slap on my various servers, but this is probably just as good an approach. I like it.
What I really also want is a way to do an offline index that this reads ... I ended up using duc. Maybe I will fork and add it!
thanks for sharing!
I had been exploring using an embedded database as an index, but for my current use case, waiting just under a minute to rescan my /nix/store on a weak mini-pc is acceptable.
Also looking to add inotify integration, which would require an index to accurately update the visualization.
I might borrow a little from that CI workflow for cross-compilation.