Skip to content

Commit b7bddcd

Browse files
schierlmschierlm
committed
First version
1 parent e663d99 commit b7bddcd

18 files changed

+1540
-0
lines changed

‎.gitignore

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
build
2+
dist

‎README.md

+190
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,192 @@
11
# usb-modboot
22
Boot multiple systems from a single GRUB2-powered USB drive (just drop ISO or other modules to integrate into menu)
3+
4+
5+
## Design Principles
6+
7+
- Modular design: You can choose yourself which modules you want to add to your USB key,
8+
so things you don't need won't need any space on your USB key. The core files (which
9+
are required in all cases) have a total size of less than 6 MB. The menu will adjust
10+
automatically as you add or remove module files.
11+
12+
- Installing USB-Modboot (making a USB key bootable for the first time) is possible in
13+
as many operating systems as possible. Currently it can be done on Windows and Linux
14+
(on Linux there are two methods depending on whether you have grub-bios-install binary
15+
available or not). Alternatively, a dd-able USB image is available that you can `dd`
16+
to the key on supported OSes, even when they are not Windows or Linux (deleting all
17+
your data on the USB key.) When only EFI boot is required (no legacy BIOS), there is
18+
no special install procedure at all - just format your USB-key as FAT32 and copy all
19+
files to it.
20+
21+
- Adding or updating modules is possible on all operating systems that can write FAT32
22+
partitions, even mobile devices with USB to go. Some modules may require unpacking .zip
23+
files, but even that shmould be possible on most devices/operating systems.
24+
25+
- ISO files of most Linux distributions (which include a loopback.cfg file), as well as
26+
EFI utilities (like EFI shell) are valid modules, so you can add them without needing
27+
any extra files. Other modules will need some extra files (available for download here),
28+
but the original (ISO etc.) files will be unchanged on the USB key to make updating
29+
(and reuing them in VMs) easier. The menu label of ISO files is tried to be determined
30+
from the content (e. g. `/.disk/info`) with a fallback derived from the file name. You
31+
can modify this fallback by adding a menu label to the file name (in square brackets)
32+
or by addng a .ini file next to the module that contains the desired menu label.
33+
34+
- When you are like me and add lots of modules, the menu can get quite cluttered.
35+
Therefore, it is possible to manually curate a favourites menu (by editing a text file
36+
on the USB key), which may also optionally include submenus. All other modules will remain
37+
in the "All Boot Modules" menu.
38+
39+
- The USB key can be booted either via legacy BIOS mode or via (64-bit) UEFI mode (including
40+
Secure Boot); some modules might not work in one of the modes, though. Also, an additional
41+
module is available to add 32-bit EFI mode (used on some tablets and netbooks), and another
42+
one to make it bootable (chainloadable) from USB-Loader floppy or CD media (in case you
43+
still find machines that cannot natively boot from USB).
44+
45+
- The USB key uses a PC (MBR) partition table and a single FAT32 formatted partition,
46+
to make it usable by as many devices as possible. It is also possible to move all
47+
files to a different disk and back later, and the USB device can still be booted.
48+
It is also possible to copy the files to a new USB key and re-run the installer, and
49+
keep all modules and menu customizations.
50+
51+
- The number of "core files" (files installed by the core) is minimized as far as
52+
possible. BIOS boot support requires 2 files (both under `/usb-modboot/` directory),
53+
efi boot support (including Secure Boot) requires 4 additional files (under `/efi/`
54+
directory). All module files are tried to be kept under these two directories, too.
55+
That way, your USB stick is kept neat and clean; and filesystem checks do not take
56+
longer than needed. There are exceptions: Windows 10 Recovery needs to have a few files
57+
under `/boot/`, for example.
58+
59+
- By default, the installer does not format your USB key or delete files from it; so you
60+
should be able to add usb-modboot to an existing USB key without having to back up your
61+
data first (unless you are using the dd-able image mentioned above).
62+
63+
64+
### Supported Modules
65+
66+
To add modules, just copy them into `/usb-loader/`directory. The following module types
67+
are supported:
68+
69+
- Directories or `tar` files containing a `grub.cfg` file. These modules will then chainload
70+
this config file. Individual `.cfg` files are also valid modules, if they are not inside of
71+
directories or `tar` files. The variable `MODULE_PATH` can be used inside the config file
72+
to point back to the module (in case the user renames it).
73+
74+
- ISO images. If they contain a `/boot/grub/loopback.cfg`, they are bootable as is. Other .iso files
75+
need a companion file with same base name, named `*.iso.module.tar` or `*.iso.module.cfg`, which
76+
needs to be a valid module and is loaded instead of the ISO file.
77+
78+
- 16-bit Linux "kernel" images (`.lkrn` files or `.bin` files). Made famous by Memtest86+ and gPXE/iPXE.
79+
80+
- 1.44 MB floppy images (`.img` files). Woll be booted by memdisk, when in BIOS mode. Cannot be booted
81+
in UEFI mode.
82+
83+
- 64-bit EFI images (`.efi` files). Can only be booted in 64-bit UEFI mode. When 32-bit UEFI support is
84+
added, 32-bit EFI images (`.efi32` files) can be used as modules in that environment.
85+
86+
- also, for every module there can be an .ini file with same name (so for `ipxe.lkrn` it has to be called
87+
`ipxe.lkrn.ini`) which can contain additional options. For now, the only supported option is `LABEL=` which
88+
can be used to override the menu label.
89+
90+
91+
## Installing usb-modboot
92+
93+
### Installing the core
94+
95+
For this, you need a USB key which contains a single partition (MBR partitioning scheme) which is formatted
96+
with FAT32 filesystem. Most USB keys are formatted like this from the factory. When formatting it yourself
97+
in Linux, choose partition type `0B` and don't forget to set the *bootable* flag (which is required by some
98+
BIOSes to make the USB key appear in the boot menu).
99+
100+
Then, copy the content of [usb-modboot.zip](https://github.com/schierlm/usb-modboot/releases/download/v0.9/usb-modboot.zip) to that USB key.
101+
On Windows, double click `\install\bios-install.cmd` file, on Linux you can change to `install` directory and run `./bios-install.sh` if you
102+
have grub-bios-setup installed (which will dynamically set up GRUB like on Windows), or run `./bios-install-from-bootblock.sh /dev/sdX` (which
103+
will use some `dd` magic to put GRUB onto the MBR). Note that the latter command will need the name of the device as an argument, while the first
104+
command (both on Windows and on Linux) will autodetect the device from where the file is stored and (after a confirmation) work automatically.
105+
106+
On other operating systems that support neither of the script, you can write the disk image `usb-bootblock.img.xz` (after extracting) onto an empty
107+
USB key with at least 1GB capacity. Then you'll have to increase the partition size of the partition to fill the full USB key, and then copy the
108+
contents on the USB key again. This is more like a last resort method; if possible I would always prefer one of the other methods.
109+
110+
After installation is finished, you may delete the `install` directory (or keep it in case you want to copy the files to another USB key later, for
111+
easier re-running).
112+
113+
114+
### Adding modules
115+
116+
Download the modules you like and copy them to `usb-modboot` directory. Modules available here are `.zip` files; they need to be extracted to the root of
117+
the USB key (but will drop the majority of files in `usb-modboot`, too). Modules will be automatically picked up when booting, so there is no need
118+
to edit menu files (unless you want to add the modules into the favourite modules menu).
119+
120+
- [module.win10recovery.zip](https://github.com/schierlm/usb-modboot/releases/download/v0.9/module.win10recovery.zip) can be used to add
121+
Windows 10 Recovery (one edition or multiple ones) to the USB key. The module comes
122+
with [RecoveryDriveBuilderPlus](https://github.com/schierlm/RecoveryDriveBuilderPlus), which you have to use to add the recovery options to your
123+
USB key. After you have added them, you may delete RecoveryDriveBuilderPlus or keep it to simplify adding more recovery drives later.
124+
125+
- [module.efitools.zip](https://github.com/schierlm/usb-modboot/releases/download/v0.9/module.efitools.zip) adds additional EFI tools
126+
(Memtest, EFI shell, and some utilities to be used in EFI shell) to the boot menu if you are booting in EFI mode.
127+
128+
- [module.efi32.zip](https://github.com/schierlm/usb-modboot/releases/download/v0.9/module.efi32.zip) adds 32-bit EFI booting
129+
support, as well as 32-bit EFI and UEFI shells.
130+
131+
- [module.netboot.zip](https://github.com/schierlm/usb-modboot/releases/download/v0.9/module.netboot.zip) contains iPXE images
132+
to boot from [netboot.xyz](https://netboot.xyz/) and [boot.fedoraproject.org](https://boot.fedoraproject.org/)
133+
134+
- [module.memtest.zip](https://github.com/schierlm/usb-modboot/releases/download/v0.9/module.memtest.zip) contains
135+
Memtest86 and Memtest86+ for BIOS mode (Memtest for EFI mode is part of efitools).
136+
137+
- [module.super_grub2_disk.zip](https://github.com/schierlm/usb-modboot/releases/download/v0.9/module.super_grub2_disk.zip) provides a small
138+
(<200KB) lite version of [Super Grub2 Disk](https://www.supergrubdisk.org/). The official Super Grub2 Disk ISO works too, but the fonts look
139+
a bit worse, and it is a lot bigger (not to forget the big scary warning at startup that loopback booting is not officially supported).
140+
141+
142+
There are also modules that should be dropped next to an [UltimateBootCD 5.3.7 ISO](http://www.ultimatebootcd.com/) or a
143+
[Debian 9.3.0 netinst ISO](https://www.debian.org/distrib/), to support booting from them. More modules may be added in the
144+
future (maybe without updating this README file).
145+
146+
147+
### Configuring favourite modules
148+
149+
When you have many modules on usb-modboot, the menu can become long and selecting may take some time. Therefore, you can add favourites which appear
150+
in the main menu (other modules will still appear in the "All Boot Modules" menu). Therefore you edit `menu.ini` and add lines like
151+
152+
addmodule ubuntu.iso
153+
154+
or
155+
156+
addmodule ubuntu.iso "My Favourite Ubuntu"
157+
158+
in case you want to use a custom title. The first argument to addmodule is the filename of the file inside `/usb-modboot/` directory
159+
(but without the directory name).
160+
161+
In fact, you can use all GRUB commands (including `submenu`) in this file, so you can build more complex favourite menus if you prefer.
162+
But you don't have to - for most users, just using addmodule should be fine.
163+
164+
165+
## Compiling usb-modboot from source
166+
167+
If you want to modify how usb-modboot works, you have to compile it from source.
168+
If you just want to use usb-modboot, you can ignore this part.
169+
170+
I use a minimal Debian Stretch VM for building; it should work on other distros as well.
171+
root privileges are required.
172+
173+
First you need some build tools:
174+
175+
# apt install build-essential unzip gettext libfreetype6-dev dosfstools autoconf automake bison flex xfonts-unifont python ca-certificates git
176+
177+
Then checkout this repo:
178+
179+
# git clone https://github.com/schierlm/usb-modboot
180+
181+
First compile grub
182+
183+
# cd usb-modboot
184+
# ./build-grub
185+
186+
Then build the main core image and some more auxiliary files
187+
188+
# ./build-main
189+
190+
The resulting files are in `dist` now. Note that this command also builds some modules;
191+
other modules have dedicated build scripts; yet other modules do not have any build scripts
192+
but are put together manually from other source pages.

‎bios-install-from-bootblock.sh

+17
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
#!/bin/sh
2+
if [ -z "$1" -o ! -b "$1" ]; then
3+
echo "Usage: sh ./bios-install-from-bootblock /dev/sd<x>"
4+
exit 1
5+
fi
6+
clear
7+
echo Install GRUB for USB-ModBoot to device $1.
8+
echo
9+
read -p "Are you sure (y/N)? " yn
10+
case $yn in
11+
[Yy]* ) break;;
12+
* ) exit;;
13+
esac
14+
echo Installing...
15+
xzcat ./usb-bootblock.img.xz | dd of=$1 bs=446 count=1 status=none
16+
xzcat ./usb-bootblock.img.xz | dd of=$1 bs=512 count=62 skip=1 seek=1 status=none
17+
echo Done.

‎bios-install.cmd

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
@echo off
2+
cd /D %~dp0
3+
for /f "usebackq" %%i in (`grub-probe -t disk %~f0`) do set DISK=%%i
4+
set LETTER=%~d0
5+
cls
6+
echo Install GRUB for USB-ModBoot to drive %LETTER% (device %DISK%).
7+
echo.
8+
choice /M "Are you sure"
9+
if not %ERRORLEVEL% == 1 exit
10+
echo Installing...
11+
grub-bios-setup -d . %DISK%
12+
echo Done.
13+
pause

‎bios-install.sh

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
#!/bin/sh
2+
DISK=`grub-probe -t disk "$0"`
3+
clear
4+
echo Install GRUB for USB-ModBoot to device $DISK.
5+
echo
6+
read -p "Are you sure (y/N)? " yn
7+
case $yn in
8+
[Yy]* ) break;;
9+
* ) exit;;
10+
esac
11+
echo Installing...
12+
grub-bios-setup -d . "$DISK"
13+
echo Done.

‎build-debian-installer-module

+52
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
#!/bin/bash -e
2+
if [ $# -lt 2 ]; then
3+
echo "Usage: ./build-debian-installer-module </path/to/filename.iso> http://<debianmirror>/debian/dists/<codename>/main/installer-<arch>/<version>/images"
4+
exit 1
5+
fi
6+
7+
rm -rf build/debian
8+
mkdir -p build/debian
9+
mount $1 build/mnt -o ro,loop
10+
VMLINUZ_NAME=$(grep vmlinuz build/mnt/boot/grub/grub.cfg | awk '{print $2}' | head -n 1)
11+
FOLDER_NAME=${VMLINUZ_NAME%/vmlinuz}
12+
13+
wget -P build/debian "$2/cdrom/vmlinuz"
14+
15+
if ! cmp -s build/debian/vmlinuz build/mnt$VMLINUZ_NAME; then
16+
echo 'Error: VMLINUZ differs on the iso - did you specify the correct debian installer version?'
17+
umount build/mnt
18+
exit 1
19+
fi
20+
21+
rm build/debian/vmlinuz
22+
wget -P build/debian "$2/hd-media/gtk/initrd.gz"
23+
mv build/debian/initrd.gz build/debian/gtk_initrd.gz
24+
wget -P build/debian "$2/hd-media/initrd.gz"
25+
26+
touch build/debian/q.gz
27+
28+
sed '0,/theme/d' build/mnt/boot/grub/grub.cfg | \
29+
sed "s#${FOLDER_NAME}/vmlinuz#${FOLDER_NAME}/vmlinuz iso-scan/filename=\${iso_path} iso-scan/ask_second_pass=true#" | \
30+
sed "s#${FOLDER_NAME}/gtk/vmlinuz#${FOLDER_NAME}/gtk/vmlinuz iso-scan/filename=\${iso_path} iso-scan/ask_second_pass=true#" | \
31+
sed "s#${FOLDER_NAME}/gtk/initrd.gz#(mod)/gtk_initrd.gz#" | \
32+
sed "s#${FOLDER_NAME}/initrd.gz#(mod)/initrd.gz#" >build/debian/debian.cfg
33+
34+
printf 'LABEL=%q\n' "$(cat build/mnt/.disk/info)" >$1.module.tar.ini
35+
umount build/mnt
36+
37+
cat >build/debian/grub.cfg <<EOF
38+
set orig_root="\$root"
39+
regexp -s 1:iso_path '^(.*)\\.module\\.tar\$' "\$MODULE_PATH"
40+
export iso_path
41+
loopback deb "\$iso_path"
42+
set root=(deb)
43+
configfile (mod)/debian.cfg
44+
loopback -d deb
45+
set root="\$orig_root"
46+
EOF
47+
48+
cd build/debian
49+
tar cf debian.tar *.cfg *.gz
50+
cd ../..
51+
mv build/debian/debian.tar $1.module.tar
52+
echo "Module build finished."

‎build-grub

+77
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
#!/bin/bash -e
2+
3+
GRUB_VERSION=2.02
4+
GRUB_VERSION_SUFFIX=-umb0.9
5+
SYSLINUX_VERSION=6.03
6+
7+
# download, unpack and patch sources
8+
rm -rf build/grub build/core dist
9+
mkdir -p build/{grub,mnt} build/core/boot/grub/{i386-pc,x86_64-efi,locale,fonts,kbd} build/core32/boot/grub/i386-efi dist/usb-modboot/efi/boot dist/usb-modboot/usb-modboot dist/usb-modboot/install dist/efi32/efi/boot
10+
cd build
11+
[ -f grub-${GRUB_VERSION}.tar.xz ] || wget http://ftp.gnu.org/gnu/grub/grub-${GRUB_VERSION}.tar.xz
12+
[ -f grub-${GRUB_VERSION}-for-windows.zip ] || wget http://ftp.gnu.org/gnu/grub/grub-${GRUB_VERSION}-for-windows.zip
13+
[ -f syslinux-${SYSLINUX_VERSION}.tar.xz ] || wget https://www.kernel.org/pub/linux/utils/boot/syslinux/syslinux-${SYSLINUX_VERSION}.tar.xz
14+
cd grub
15+
tar xfvJ ../grub-${GRUB_VERSION}.tar.xz
16+
tar xfvJ ../syslinux-${SYSLINUX_VERSION}.tar.xz syslinux-${SYSLINUX_VERSION}/bios/memdisk/memdisk
17+
unzip ../grub-${GRUB_VERSION}-for-windows.zip grub-${GRUB_VERSION}-for-windows/grub-{probe,bios-setup}.exe
18+
mv grub-${GRUB_VERSION}-for-windows/*.exe ../../dist/usb-modboot/install
19+
mv syslinux-${SYSLINUX_VERSION}/bios/memdisk/memdisk ../core/boot/grub
20+
cd grub-${GRUB_VERSION}
21+
patch -p1 <../../../grub.patch
22+
./autogen.sh
23+
24+
# build i386-efi image
25+
./configure PACKAGE_VERSION="${GRUB_VERSION}${GRUB_VERSION_SUFFIX}" PACKAGE_STRING="GRUB ${GRUB_VERSION}${GRUB_VERSION_SUFFIX}" --with-platform=efi --target=i386-pe
26+
make
27+
echo 'loopback core32 ($root)/efi/boot/core32.dat' >../confscript_efi32
28+
echo 'set prefix=(core32)/boot/grub' >>../confscript_efi32
29+
./grub-mkimage -d ./grub-core -O i386-efi -o ../../../dist/efi32/efi/boot/bootia32.efi -p '' -c ../confscript_efi32 fat part_msdos loopback
30+
cp grub-core/*.{lst,mod} ../../core32/boot/grub/i386-efi
31+
32+
# build x86_64-efi image
33+
./configure PACKAGE_VERSION="${GRUB_VERSION}${GRUB_VERSION_SUFFIX}" PACKAGE_STRING="GRUB ${GRUB_VERSION}${GRUB_VERSION_SUFFIX}" --with-platform=efi --target=amd64-pe
34+
make
35+
echo 'loopback core ($root)/usb-modboot/core-module.dat' >../confscript_efi
36+
echo 'set prefix=(core)/boot/grub' >>../confscript_efi
37+
./grub-mkimage -d ./grub-core -O x86_64-efi -o ../../../dist/usb-modboot/efi/boot/grub.efi -p '' -c ../confscript_efi fat part_msdos loopback
38+
cp grub-core/*.{lst,mod} ../../core/boot/grub/x86_64-efi
39+
40+
# build i386-pc image
41+
make distclean
42+
./linguas.sh
43+
./configure PACKAGE_VERSION="${GRUB_VERSION}${GRUB_VERSION_SUFFIX}" PACKAGE_STRING="GRUB ${GRUB_VERSION}${GRUB_VERSION_SUFFIX}" --with-platform=pc
44+
make
45+
echo 'set root=$root,msdos1' >../confscript_bios
46+
echo 'loopback core ($root)/usb-modboot/core-module.dat' >>../confscript_bios
47+
echo 'set prefix=(core)/boot/grub' >>../confscript_bios
48+
./grub-mkimage -d ./grub-core -O i386-pc -o ../../../dist/usb-modboot/install/core.img -p '' -c ../confscript_bios biosdisk fat part_msdos loopback
49+
cp grub-core/*.{lst,mod} grub-core/efiemu{32,64}.o ../../core/boot/grub/i386-pc
50+
cp grub-core/boot.img ../../../dist/usb-modboot/install
51+
52+
# build USB-Loader compatible i386-pc image
53+
echo "echo Core loaded from USB. Press [Return] to boot." >../usbloaderscript
54+
echo "read tmp" >>../usbloaderscript
55+
echo "search.file /usb-modboot/core-module.dat root" >>../usbloaderscript
56+
echo 'loopback core ($root)/usb-modboot/core-module.dat' >>../usbloaderscript
57+
echo 'set prefix=(core)/boot/grub' >>../usbloaderscript
58+
./grub-mkimage -d ./grub-core -O i386-pc -o ../usb-loader-core.img -p '' -c ../usbloaderscript at_keyboard usb_keyboard usbms uhci ohci ehci fat part_msdos loopback search_fs_file read echo
59+
60+
# copy generic files
61+
cp po/de.gmo ../../core/boot/grub/locale
62+
cp euro.pf2 ../../core/boot/grub/fonts
63+
cp docs/osdetect.cfg ../../core/boot/grub
64+
for i in de at ch es fr us cz; do
65+
ckbcomp $i | ./grub-mklayout -o ../../core/boot/grub/kbd/$i.gkb
66+
done
67+
68+
# patch MBR to avoid detecting it as FAT-formatted superfloppy by some embedded devices
69+
# (see <https://github.com/schierlm/grml-plus/blob/master/data/patch-grub-mbr.pl#L3-14>)
70+
xxd -r - ../../../dist/usb-modboot/install/boot.img <<'EOF'
71+
0000000: 90eb 62
72+
EOF
73+
xxd -r - ../../../dist/usb-modboot/install/core.img <<'EOF'
74+
000011b: 5553 4220 4d42 52
75+
EOF
76+
77+
echo "GRUB build finished."

0 commit comments

Comments
 (0)