Osx86 USB Maker and Utilities for Linux (Without a MAC)

[smx]

Hi, i started a project to bring the task of creating bootable usb on Linux, in a script form.

It's similar to Pandora, but i wrote it from scratch. As of now it lacks a GUI, and i lack GUI coding skills .

It can be usefuel for:

 

-People without access to a real MAC.

-People without Virtualization support in CPU for VM

-People with their OS X installation unbootable (you can avoid installing in VM to create Installer, you can create create Installer directly and faster).

-People needing to restore their original MAC (not possible yet, in TODO list, i'd need a tester and i'd probably need work with GUID partitioning. I don't know if real MACs can boot MBR USB).

-People who want to do it with Linux because they just want to .

 

The script is currently in testing, if you want to try it out please understand i'll take no responsibility for wiped hard drives or data loss, but i'll help you as i can if you experience any problem.
However, i made several tests on both Virtual Machines and real Hardware (and different PCs) with no problem at all, using 10.6/10.7/10.8/10.9 dmg files.

Features:

• Creates a bootable installation usb drive, starting from dmg files (InstallESD.dmg or Mac OS X Install DVD.dmg). CD to USB is implemented, but not tested.
• Creates osx86 installation HDD images starting from dmg file, those are suitable for VMs. (SEE NOTES BELOW)
• Extracts a pkg or mpkg file (using "xar", which is self-compiled by the script)
• Converts dmg files to img (using "dmg2img", which is self-compiled by the script)
• After the media is ready, you can re-run the tool to access the management menu. It allows:
  • Kext Management
  • Chameleon Modules Management
  • Kernel Management
  • Chameleon repair/update
  • (Re)apply MBR patch
  • (Re)install Smbios
  • (Re)install DSDT
  • Erase Setup (to wipe ALL the usb data and restore the drive to normal use)
• Logging to html, with ansi2html utility by running "install_osx_log.sh" with same parameters you'd use with normal one

USAGE:

NOTE: all the commands below must be used without [] charachers. [] indicate you have to change the content according to your needs and specific case!!

NOTE: use ./install_osx_quiet.sh to avoid really verbose output.

 

How to setup files:

--Chameleon--

You need obtain the following chameleon core files.

• boot0
• boot1h
• boot

Place them in "chameleon" directory. You can either get those files from "i386" archives, or download the pkg and unpack it

sudo ./install_osx.sh <package.pkg> <destdir>

Where outputdir is a temporary location (not the "chameleon" folder!).

You'll find core files in /Core.pkg/usr/standalone/i386. Move the required files in "chameleon" directory.
Nolw you can delete as we don't need it anymore

 

Chameleon dylib modules go to "chameleon/Modules" (example Sata.dylib, Resolution.dylib, etc...).

You can manage them with "Manage chameleon Modules" menu option, in case you need to add/remove them later.

 

Chameleon Themes go to "chameleon/Themes" (chameleon will use the theme in folder named "Default" at boot, "chameleon/Themes/Default")

 

--Kexts--

Place any kext (they are seen as directories on Linux) in "extra_kexts" directory.

You can manage them with "Manage kexts" menu option, in case you need to add/remove them later.

 

--MBR Patch--

Place the following files in "osinstall_mbr" directory

• OSInstall
• OSInstall.mpkg

They NEED to match the version on DMG, and the script will check that before copying. I plan to add a way to allow multiple filesets for each installer version.

 

--Kernels--

Place them in "kernels" directory. They are not automatically installed, you need to invoke the management menu to swap mach_kernel or add a new kernel to root.
As of now kernels are copied as is. To replace main kernel you need to rename the desired kernel to "mach_kernel".

This allows multiple kernels on the same installation media.

 

Example:

If you leave the filenames as is and, let's say you have a file named "cust_kernel", you'll have to boot with: "cust_kernel" at chameleon prompt.

 

--Tweaks--

Single tweaks, stored in "tweaks" folder, that can extend the script (i plan to convert some internal function to tweak to make it more light-weight)

Example of tweak:

"Remove Graphic Kexts"

"Restore Graphic Kexts"

 

A tweak is like a plugin. If you want to add a tweak (if you are a developer), you must add the string

tweakname="My Tweak"

So that it will appear in the menu as "My Tweak" in the above example.

A tweak can use functions and variables exported from the main script (with "export")

 

 

 

How to extract a pkg/mpkg:

sudo ./install_osx.sh [pkgfile] [destdir]

How to prepare OSX86 usb

run

sudo blkid

To obtain info about your USB Drive (you need the id, Example /dev/sde)

sudo ./install_osx.sh [dmg path] [/dev/sdX]

where X is the letter you gathered above

 

How to prepare OSX86 image

sudo ./install_osx.sh [dmg path] [dest.img]

How to manage images and usb drives (management menu)

For USB drives:

sudo ./install_osx.sh [/dev/sdX]

For images

sudo ./install_osx.sh [imagefile]

DOWNLOAD and PREPARATION:

You can get the script from https://github.com/smx-smx/osx86_linux, using git

 

LATEST VERSION: r23

git clone https://github.com/smx-smx/osx86_linux.git 

OR: You can use the github "Download ZIP" button

 

To check for script updates, just do

git pull

In script directory.

 

INSTALL THE FOLLOWING DEPENDENCIES: (post if i forgot a dep)

On Ubuntu/Mint:

sudo apt-get install cpio hfsprogs tput wget parted udisks qemu-utils build-essential zlib1g-dev
sudo apt-get build-dep dmg2img

I tested it on Arch Linux aswell, but i don't remember the complete set of dependencies i installed. if i recall correctly:

udisks - tput - hfsprogs(AUR) - wget - parted - cpio - qemu - base-devel - zlib

 

If you encounter  a regression in my script, you can downgrade with

git reset --hard HEAD~[number]

Where [number] is the number of updates to rollback, e.g 1 to return to previous version, 2 to roll-back 2 updates and so on. (please post any regression or bug)

 

NOTES

WARNING: during my tests i found out the best deal is "img" raw format. If you specify VHD/VMDK/VDI your system may freeze or crash. This is due to software limitations on Linux. To work around this problem, i strongly recommend using "img" output if you need Installation Hard Disk images. The script will ask you if you want to convert to VDI/VMDK/VHD at the end of the process, after all the data has been flushed (Virtualbox required, as it contains the "vboxmanage" command, needed to convert img -> something_else).
Looks like the main cause of this problem is combining the FUSE NTFS driver with VDI/VMDK/VHD. You can try VDI/VHD/VMDK and post your result. I'll edit these warnings accordingly.
Using the Management Menu with an already created VHD/VDI/VMDK should be fine, as the data to be written is a lot smaller (as osx files are already there)

 

WARNING: When outputting the img, it's better to choose a drive with ext filesystem. AVOID NTFS DRIVES, they may perform very very slow or the script may hang. This is due to the ntfs-3g fuse driver. You can choose an ext4 drive, and move the resulting image at the end of the process.

 

WARNING: Some distributions may spit out errors during cp with "cp preserving permissions not supported". I have that problem on ArchLinux, but not on Mint. I suspect there's a regression in linux hfs+ driver. Anyway it should still work fine.

 

NOTICE: You need a lot of disk space for this script.

Example for USB:
InstallESD.dmg(5.3G) + InstallESD.img(5.4G) + BaseSystem.img(1.3G) = approx 12GB (only for setup data)

Example for VDI:

InstallESD.dmg(5.3G) + InstallESD.img(5.4G) + BaseSystem.img(1.3G) + mavericks.img(10G) + mavericks.vdi(6G) = approx 28GB (only for setup data)(18 after removing mavericks.img after conversion)

 

The img files are temporary! The script doesn't delete them so it can create USB pendrives faster the next time you run it. If you plan to use it once, or if you need to free some disk space, you CAN DELETE *.img and leave only the dmg file you started with.

Here's a look at the finished product, maverick with img converted to vdi option  

 

Real HW, Snow Leopard 10.6

 

Installers prepared from scratch with Linux and this script only.

[tlac]

What is your trick to using the dmg2img? When I played with that it didn't work for me with Mavericks.

[smx]

Building the latest one from source (1.6.5). I get problems with older ones aswell.
The script downloads and builds it automatically, same for xar. That's done in compile_d2i() and compile_xar().

With this method permissions are preserved. I prepare an hfs+ partition without journal (journal not supported on Linux), and i copy all the data in there

[tlac]

Weird. I also tried that version.

 

Some notes:

 

They NEED to match the version on DMG, and the script will check that before copying. I plan to add a way to allow multiple filesets for each installer version.

 

Instead of the filesets you can write (or use) a tool that patch the files.

 

Here's a look at the finished product, maverick with img converted to vdi option

 

It is not really needed to convert it for virtualbox. Just use the '.hdd' extension for the raw image.

 

git clone https://github.com/smx-smx/osx86_linux.git

 

Could you do a portable version that contains the scripts and the dependencies (in statically compiled way)? It could be more user friendly.

For example if the user just start a live linux and download your package, unpack it, run it, everything will work.

[smx]

I didn't know virtualbox supports .img files as .hdd, i used vdi as it uses less space btw.
About the compiling yes, i can bundle dmg2img and xar already built (i'll do that), but dependencies such as qemu_utils, cpio, tput and hfsprogs will still be needed. The script is interpreted by bash without the need to be built. You can get a zip from GitHub "Download ZIP" button without using git.

Pandora uses filesets aswell, i just looked into it.
To avoid them i'd need to diff patch -> original, and look what instruction (a jump i suppose) i need to change by doing an AOB (Array of Bytes) scan. I'll look at that

[midi-sama]

Thanks a lot

[sylreboux]

Hi,

 

Thanks for the script! I am trying to convert a dmg file from OSX Lion to an img file. I'm having an error when it tries to mount esd.

I am on Linux Mint 14.

 

Here is the output I get:

~/osx86_linux (master)$ sudo ./install_osx.sh ~/OSXLION/Contents/SharedSupport/InstallESD.dmg InstallESD.img
OSX Install Media Maker by SMX
Version: r23
Setting qemu-nbd dev...
Virtual HDD Image Mode
qemu-nbd: Found
FreeSpace:    13924327424
Needed:        10737418240
Checking Commands...
udisks: Found
grep: Found
tput: Found
dd: Found
sed: Found
parted: Found
awk: Found
mkfs.hfsplus: Found
wget: Found
dirname: Found
basename: Found
parted: Found
pidof: Found
gunzip: Found
bunzip2: Found
cpio: Found
Creating Image...
0+0 records in
0+0 records out
0 bytes (0  copied, 7.086e-06 s, 0.0 kB/s
Creating Partition Table on InstallESD.img...
Creating new Primary Active Partition on InstallESD.img
Model:  (file)
Disk /home/sylvain/osx86_linux/InstallESD.img: 10.7GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  10.7GB  10.7GB  primary

Mapping virtual dev with qemu...
Formatting partition as HFS+
Initialized /dev/nbd0p1 as a 10239 MB HFS Plus volume
Mapping image with qemu...
Mounting Partitions...
mount: special device /dev/nbd1p2 does not exist
mount: special device /dev/nbd1p2 does not exist
mount: special device /dev/nbd1p3 does not exist
mount: special device /dev/nbd1p3 does not exist
Cannot mount esd

Any idea what is going on?

[smx]

I'll need to fixup it a little, but meanwhile try giving me the output of

$ 7z l

[sylreboux]

I'll need to fixup it a little, but meanwhile try giving me the output of

$ 7z l <drag your DMG file>

Thanks! Here is the output:

7-Zip [64] 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,2 CPUs)

Listing archive: InstallESD.dmg

--
Path = InstallESD.dmg
Type = Xar

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
1970-01-01 01:00:00 D....                            Resources
1970-01-01 01:00:00 D....                            Resources/ar.lproj
2011-06-30 04:28:08 .....        24394         7061  Resources/ar.lproj/Localizable.strings
2011-06-30 04:28:08 .....          184          143  Resources/ar.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/cs.lproj
2011-06-30 04:26:46 .....        29108         7767  Resources/cs.lproj/Localizable.strings
2011-06-30 04:26:46 .....          210          152  Resources/cs.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/da.lproj
2011-06-30 05:02:23 .....        26685         7332  Resources/da.lproj/Localizable.strings
2011-06-30 05:02:23 .....          147          125  Resources/da.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/Dutch.lproj
2011-06-30 05:02:33 .....        23585         7453  Resources/Dutch.lproj/Localizable.strings
2011-06-30 05:02:33 .....          147          128  Resources/Dutch.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/English.lproj
2011-06-30 04:58:10 .....        18533         6756  Resources/English.lproj/Localizable.strings
2011-06-30 04:58:10 .....          125          113  Resources/English.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/fi.lproj
2011-06-30 05:00:27 .....        25576         7445  Resources/fi.lproj/Localizable.strings
2011-06-30 05:00:27 .....          192          136  Resources/fi.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/French.lproj
2011-06-30 05:02:39 .....        35089         8049  Resources/French.lproj/Localizable.strings
2011-06-30 05:02:39 .....          222          154  Resources/French.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/German.lproj
2011-06-30 05:02:53 .....        26940         7636  Resources/German.lproj/Localizable.strings
2011-06-30 05:02:53 .....          145          128  Resources/German.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/hu.lproj
2011-06-30 04:27:47 .....        31901         7897  Resources/hu.lproj/Localizable.strings
2011-06-30 04:27:47 .....          264          176  Resources/hu.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/Italian.lproj
2011-06-30 05:00:19 .....        22154         7300  Resources/Italian.lproj/Localizable.strings
2011-06-30 05:00:19 .....          134          118  Resources/Italian.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/Japanese.lproj
2011-06-30 05:00:03 .....        20601         6854  Resources/Japanese.lproj/Localizable.strings
2011-06-30 05:00:03 .....          172          154  Resources/Japanese.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/ko.lproj
2011-06-30 04:59:56 .....        19260         6900  Resources/ko.lproj/Localizable.strings
2011-06-30 04:59:56 .....          146          129  Resources/ko.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/no.lproj
2011-06-30 05:03:20 .....        27248         7465  Resources/no.lproj/Localizable.strings
2011-06-30 05:03:20 .....          200          138  Resources/no.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/pl.lproj
2011-06-30 05:04:05 .....        34541         8076  Resources/pl.lproj/Localizable.strings
2011-06-30 05:04:05 .....          218          153  Resources/pl.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/pt.lproj
2011-06-30 05:02:27 .....        28632         7699  Resources/pt.lproj/Localizable.strings
2011-06-30 05:02:27 .....          280          172  Resources/pt.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/pt_PT.lproj
2011-06-30 05:00:08 .....        28622         7608  Resources/pt_PT.lproj/Localizable.strings
2011-06-30 05:00:08 .....          242          156  Resources/pt_PT.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/ru.lproj
2011-06-30 05:02:26 .....        31310         7963  Resources/ru.lproj/Localizable.strings
2011-06-30 05:02:26 .....          196          149  Resources/ru.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/Spanish.lproj
2011-06-30 05:02:49 .....        31287         7827  Resources/Spanish.lproj/Localizable.strings
2011-06-30 05:02:49 .....          256          157  Resources/Spanish.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/sv.lproj
2011-06-30 05:00:18 .....        28529         7488  Resources/sv.lproj/Localizable.strings
2011-06-30 05:00:18 .....          220          150  Resources/sv.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/tr.lproj
2011-06-30 04:29:51 .....        29819         7557  Resources/tr.lproj/Localizable.strings
2011-06-30 04:29:51 .....          230          159  Resources/tr.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/zh_CN.lproj
2011-06-30 05:00:25 .....        15772         6604  Resources/zh_CN.lproj/Localizable.strings
2011-06-30 05:00:25 .....          122          108  Resources/zh_CN.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            Resources/zh_TW.lproj
2011-06-30 05:00:12 .....        15654         6548  Resources/zh_TW.lproj/Localizable.strings
2011-06-30 05:00:12 .....          118          104  Resources/zh_TW.lproj/VolumeCheck.strings
1970-01-01 01:00:00 D....                            InstallMacOSX.pkg
2011-06-30 19:45:50 .....       247954        44076  InstallMacOSX.pkg/Bom
2011-06-30 19:45:51 .....      5315194      5315194  InstallMacOSX.pkg/Payload
2011-06-30 19:45:51 .....         1669         1669  InstallMacOSX.pkg/Scripts
                    .....         2210          563  InstallMacOSX.pkg/PackageInfo
2011-06-30 09:09:45 .....   3737470542   3737470542  InstallMacOSX.pkg/InstallESD.dmg
2011-06-30 09:41:35 .....         7866         2597  Distribution
                    .....        56061        56061  [TOC].xml
------------------- ----- ------------ ------------  ------------------------
                            3743680906   3743057089  51 files, 24 folders

[Allan]

Very good! 

[lizbeth]

this looks good.  Havent tried it yet.  Deosnt see to difficult.  I wonder though how to edit the usb after creating it, if say it doesnt install and i need to get at some kexts for example?

[smx]

I'm sorry if this tool hasn't been updated, but i've been busy lately.
Yes, it doesn't work for Yosemite onwards (yet )
I hope i can get back to this sooner or later and update it but i could need a rewrite of the code (or parts of it), as it's a bit hacky/messy in some areas.

@lizbeth: the script has an edit mode, you can re-run it with the device it was installed to as second argument, and it will provide an edit menu, like this:

sudo ./install_osx.sh /dev/sdXY

where sdXY is the device the installer was created on. You should see something like this

Choose an operation...
1  - Manage kexts
2  - Manage chameleon Modules
3  - Manage kernels
4  - Reinstall / Update chameleon
5  - Install / Reinstall MBR Patch
6  - Install / Reinstall Custom DSDT
7  - Install / Reinstall SMBios
8  - Erase Setup
9  - Delete Kext Cache
10 - Tweaks Menu
0 - Exit

[lizbeth]

Sweet!

[Samirdu91]

Thank's man i wait the new version ^^

[tuxuser]

Wow, awesome work! Finally somebody created such nice application!