Sidebar can't cope with large projects

For large projects the sidebar folder scan seems to leave the folder/file tree incomplete. Often, folders at a certain depth remain empty, even though on disk they contain files and subfolders.

At first I thought this was because the scan had not finished, but days later this is still the case.

Other issues relating to folder scanning are
- no indicator to show when a scan is in progress or when a branch is incomplete
- no option to refresh a folder after contents were changed outside of the editor
- switching projects seems to trigger a fresh scan, which is not practical for large projects

'large', in this case is 150,000 files across 25,000 folders. Maybe sublime is not designed for projects of that size? Eclipse manages it, although the scan is pretty slow.