8 replies to this topic

[geek1337]

Hello world!

I'm new to this forum, so please excuse me if i err in some way. This is a complex question and as such requires a bit of explanation. This problem evolved from musings of adding an OptiBay enclosed SSD to my MacBook.

Abstract:
The ultimate goal of this project is two have my system files mirrored across two disk (HDD & SSD) whilst recognized as one logical partition by OSX. I want to read and write to just the SSD (since it's the fastest), and then have any changes synced two a mirror on the HD. SSD may be removed from time to time, so i want OSX to be able to boot from the mirror whenever the SSD is not found at boot (hence the single logical partition).

Possible Implementations:

• My current hypothesis as to how i might accomplish this is via a Software RAID 0. It's not that simple though, since a plain RAID 0 between the two disks would be impractical. The RAID would be limited by the read/write speed of the HD, thus negating the speed boost of the SSD.
  
  • How can set it up as a RAID 0, but only r/w to/from the SSD (from the system's perspective) and let the RAID handle syncing the data?
    
  • What would i need, in terms of software, to set up that RAID?
  
  
• An alternate method might be to set up two independent partitions (one on each disk), identify the SSD as the start up disk, and run a script (with lowered priority) that syncs the SSD partition with the former partition.
  
  • How would i go about writing such a script? What language would it have to be in to run with the smallest resource consumption?
    
  • How would i make the mirror HDD partition hidden during normal operations, but automatically boot from it when the SSD is not found?

Requirements:

• Any solutions must be a software solution and/or use hardware that would fit inside the MacBook's case
  
• The mirror partition on the HDD does not cover the whole disk. Other partitions on the HDD must be accessible at all times to the OS.
  
• The same goes for the SSD except for when it's removed.
  
• Syncing between disks should not create any appreciable latency.
  
• Restricted to the capabilities of OSX 10.6.

Thanks a bunch in advance for any help that can be provided!

[rorydaredking]

geek1337, on Sep 29 2011, 07:07 PM, said:

Hello world!

I'm new to this forum, so please excuse me if i err in some way. This is a complex question and as such requires a bit of explanation. This problem evolved from musings of adding an OptiBay enclosed SSD to my MacBook.

Abstract:
The ultimate goal of this project is two have my system files mirrored across two disk (HDD & SSD) whilst recognized as one logical partition by OSX. I want to read and write to just the SSD (since it's the fastest), and then have any changes synced two a mirror on the HD. SSD may be removed from time to time, so i want OSX to be able to boot from the mirror whenever the SSD is not found at boot (hence the single logical partition).

Possible Implementations:

• My current hypothesis as to how i might accomplish this is via a Software RAID 0. It's not that simple though, since a plain RAID 0 between the two disks would be impractical. The RAID would be limited by the read/write speed of the HD, thus negating the speed boost of the SSD.
  
  • How can set it up as a RAID 0, but only r/w to/from the SSD (from the system's perspective) and let the RAID handle syncing the data?
    
  • What would i need, in terms of software, to set up that RAID?
  
  
• An alternate method might be to set up two independent partitions (one on each disk), identify the SSD as the start up disk, and run a script (with lowered priority) that syncs the SSD partition with the former partition.
  
  • How would i go about writing such a script? What language would it have to be in to run with the smallest resource consumption?
    
  • How would i make the mirror HDD partition hidden during normal operations, but automatically boot from it when the SSD is not found?

Requirements:

• Any solutions must be a software solution and/or use hardware that would fit inside the MacBook's case
  
• The mirror partition on the HDD does not cover the whole disk. Other partitions on the HDD must be accessible at all times to the OS.
  
• The same goes for the SSD except for when it's removed.
  
• Syncing between disks should not create any appreciable latency.
  
• Restricted to the capabilities of OSX 10.6.

Thanks a bunch in advance for any help that can be provided!

Only thing I can think of is using Carbon Copy Cloner and schedule an update image every few hours/once per day/whatever, and of SSD onto HDD. You could stop HDD mounting at boot (look on google, forgot how) but don't know if CCC can mount a drive before mounting it.
Out of interest, why do you mind about 2 identical drives appearing?

[geek1337]

rorydaredking, on Oct 2 2011, 05:52 AM, said:

Only thing I can think of is using Carbon Copy Cloner and schedule an update image every few hours/once per day/whatever, and of SSD onto HDD. You could stop HDD mounting at boot (look on google, forgot how) but don't know if CCC can mount a drive before mounting it.
Out of interest, why do you mind about 2 identical drives appearing?

Thanks for the reply. I had already thought of using CCC; it might be what i eventually resort to, but i wanted to see if there was a better solution first. The problem for me with having two independent, identical drives, is that it would complicate things a fair bit (though i'll admit maybe less than other solutions).

• I'd need to write a script to mount/unmount the HDD partition before/after each pass of CCC.
  
• Not having the HDD mounted most of the time would mean that i wouldn't be able to index it.... Although, if it has the same .Spotlight-V100 directory on it (as part of the cloning), then maybe i wouldn't need to?
  
• I don't think CCC would work as a two-way sync, which it would need to be whenever i had to update the SSD from the HDD.
  
• I might be able to sidestep the last one by writing a script with dd (of maybe rsync) to do the copying. However, i don't have the kind of skill or knowledge necessary to that without messing up my system a few times first.

Maybe writing a customized script is the only solution?

[bonestonne]

There would be zero benefit of having this against say an actual RAID setup.

Like most other hardware combinations, it would only be as fast as the slowest drive in the system, but continuously updating an entire partition over and over and over could kill the drive in a matter of months, depending on how much data ends up being there.

You're better off using Time Machine and an external RAID array for speed...at least from a logistical point of view.

[geek1337]

bonestonne, on Oct 4 2011, 05:45 PM, said:

There would be zero benefit of having this against say an actual RAID setup.

Like most other hardware combinations, it would only be as fast as the slowest drive in the system, but continuously updating an entire partition over and over and over could kill the drive in a matter of months, depending on how much data ends up being there.

You're better off using Time Machine and an external RAID array for speed...at least from a logistical point of view.

Thank you. I do understand that an actual RAID would be impractically slow in this set up. I was wondering more if a RAID-like arrangement could be created. All system reads and writes would be to one drive, and syncing the two disks would occur during system down time (whenever extra resources were available). It is similar in that the two disks are mirrored and the pair is seen as one logical drive by the system, but different in that all reads and writes go to one disk and mirroring the two takes place afterwards in system down time.

Using TM with an external RAID would be fine if this were a desktop arrangement, but it's not. The goal (as i outlined initially) is that, "[a]ny solutions must be a software solution and/or use hardware that would fit inside the MacBook's case" (emphasis added).

I agree that the continuous updating of an entire partition, would kill a disk very quickly. However, a series of small, incremental changes to the disk, would shorten a drive's life expectancy no more than regular disk use would.

It is becoming evermore clear that i am perhaps talking about something that doesn't exist currently in any established form. I'm thinking that maybe rorydaredking had the right idea. I'm still unsure as to how i could implement it though. Any suggestions (script-wise)?

[bonestonne]

While updating small files at a time would not affect drive lifespan, CCC and other partition copying software do not just check for changed or new files, most simply copy the entire partition because it's much faster than searching through the filesystem for changes.

I think mixing a traditional HDD and a SSD in the way you're describing is just not going to work out in the way you want it to. Either two HDD or two SSD, but not one of both. I would simply opt for time machine more than anything else, because if it's set up correctly on an external array, the backups will be fast, as well as allowing you to go back to specific points in time, rather than just a set system image of once a day (depending on Time Machine's configuration).

From a cost standpoint, device lifespan, as well as ease of use, this is probably your best option. I work at a small computer shop, and honestly, if a customer came in with your question, that's probably what I would suggest to them. I would also suggest going with 10k RPM HDD units as opposed to SSDs specifically for the data recovery aspect, it's much easier to do on a traditional HDD than a SSD.

[geek1337]

Thank you bonestonne.You've been a lot of help. Just a few things:

bonestonne, on Oct 5 2011, 04:58 PM, said:

While updating small files at a time would not affect drive lifespan, CCC and other partition copying software do not just check for changed or new files, most simply copy the entire partition because it's much faster than searching through the filesystem for changes.

You were probably talking more about the "other partition copying software", but i thought i should point out that on the correct settings, CCC actually does scan for changes. Also, 'SuperDuper!', a similar piece of software (the popular alternative to CCC), has a 'Smart Update' feature that does the same thing.

Not that I'm an expert or anything, but from what experience i do have, i would say that a straight copy (without scanning) is only faster when you're updating a small amount of data. In any case, this isn't an operation that needs to be performed often (though it's quicker each time if it does).

  

bonestonne, on Oct 5 2011, 04:58 PM, said:

I think mixing a traditional HDD and a SSD in the way you're describing is just not going to work out in the way you want it to. Either two HDD or two SSD, but not one of both. I would simply opt for time machine more than anything else, because if it's set up correctly on an external array, the backups will be fast, as well as allowing you to go back to specific points in time, rather than just a set system image of once a day (depending on Time Machine's configuration).

A TM would be a good solution, except that it can't be booted from. It would also be redundant, since i already have a terabyte dedicated to TM. I apologize for not saying this before, but the purpose of the HDD mirror is not as a backup. It exists solely so that i can boot from it whenever the SSD is removed. It doesn't really need to be updated every day, just so long as it's up to date when i pull the SSD. It doesn't need to sync both ways all the time, just when i stick the SSD back in and have to update it from the HDD.

Maybe i should just abandon this whole thing and just use CCC whenever i need to update and write a script to unmount the HDD partition whenever it's not needed?

[bonestonne]

I'm just confused as to why you would ever remove the HDD from your computer, in order to need the SSD in the first place.

[geek1337]

bonestonne, on Oct 6 2011, 11:29 PM, said:

I'm just confused as to why you would ever remove the HDD from your computer, in order to need the SSD in the first place.

Oh, sorry for the confusion. As i said in my original statement, this project involves the installment of a SSD within an OptiBay enclosure. The procedure involves removing the optical disc and replacing it with a SSD or HDD (in this case the former). This will net me two physical drive slots, one HDD and one SSD. I plan to put my system files on the SSD and leave the rest on the HDD. For other reasons, i will need to periodically disconnect the SSD. When this happens i would like to still be able to boot the MacBook and for it to be where i left it when i do so. This necessitates a mirror partition on the HDD of the system files stored on the SSD.

I hope that explanation proved useful. I'm thinking more and more that a cloning script is the way to go.