diff --git a/README.md b/README.md index b53610d..79fb7de 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ This is my setup for my media center. Its main purpose is for me to be able to replicate it, in case of need. However, with a few modifications, it can be a great way for you to have a Raspberry Pi with the following features: - an mpd server to stream local music, with an equalizer and snapcast support to stream. - an Airplay capable machine (for Apple Music for example). -- a place to stream videos through Samba and NFS. +- a place to organize and stream videos through Samba and NFS. ## Prerequisites @@ -13,16 +13,16 @@ This is my setup for my media center. Its main purpose is for me to be able to r ## First configuration * Insert the SD card in the Pi, start up, and SSH to it. -* Be sure to be in the wanted distribution target. For instance, I regularly need to be in _testing_, for mpd to be up to date. This can be done my modifying _/etc/apt/sources.list_. I usually type in the explicit name of testing (currently _bookworm_) to be sure where I'm at. +* Be sure to be in the wanted distribution target. For instance, I regularly need to be in _testing_, for mpd to be up to date. This can be done my modifying _/etc/apt/sources.list_. I usually type in the explicit name of testing (currently _trixie_) to be sure where I'm at. * Run `sudo apt update && sudo apt dist-upgrade && sudo apt upgrade` to update the system to the last version we need. * Run `sudo apt install mpd mpc libasound2-plugin-equal shairport-sync snapserver samba nfs-kernel-server vim` to install the needed packages. ## Copying the SD card data to the USB media -Please note that this step can be done after the other configurations are performed, as most of the things will be copied. +Please note that this step can be done after the other configurations are performed, as most of the things will be copied from the SD card to the USB media device. * Transfer the utilities folder to your Pi. -* Inside, there is a folder called usb-boot. It comes from RonR, a user from the Raspberry Pi forums. You can also [download it here](https://forums.raspberrypi.com/viewtopic.php?t=196778). The version included in this repository is from Feb 28, 2022. +* Inside, there is a folder called usb-boot. It comes from RonR, a user from the Raspberry Pi forums. You can also [download it here](https://forums.raspberrypi.com/viewtopic.php?t=196778). The version included in this repository is from Feb 11, 2024. * Run `usb-boot.sh`. * You can now remove your SD card and boot the system from the USB media device. @@ -50,25 +50,23 @@ My music is copied with rsync through my NAS. For that purpose I use a set of sc * Fist, we create a folder where we'll mount to NAS: `mkdir /media/mantis-music`. * Then a folder where to copy the music: `mkdir /home/pi/music`. -* Use to script to copy locally the music: `utilities/update-music.sh`. -* Copy the mpd configuration file in `/etc/mpd.conf`. +* Use to script to copy locally the music: `utilities/1-update-music.sh`. +* Edit `/etc/mpd.conf` and add the content from this repository. * Run `mpc update`, then `mpc ls` to try to play something with `mpc play`. * Test right away, it should output sound if you've rightly configured your audio. -## Configuring the sharing +### Going further + +* You can extract the cover of audio files and uploading them to your NAS by running `utilities/2-cover-upload.sh`. +* You can remove unwated files (such as `.DS_Store` and `Thumbs.db`) and uploading these changes to your NAS by running `utilities/3-remove-unwanted-files.sh`. + +## Configuring sharing ### Samba I use Samba to be able to organize my videos through my Windows machine. -* Edit `/etc/samba/smb.conf` and add the following: - ``` - [VideoPi] - comment = Videos on Pi - path = /home/pi/video - read only = no - browsable = yes - ``` +* Edit `/etc/samba/smb.conf` and add the content from this repository. * Then set up a password for the user pi: `sudo smbpasswd -a pi` * You should be able to connect to it. @@ -76,16 +74,30 @@ I use Samba to be able to organize my videos through my Windows machine. NFS is used to communicate with my Linux machine reading videos through Kodi. -* Edit `/etc/exports` and add the following line to make it visible to the home network: `/home/pi/video 192.168.1.0/24(rw,all_squash,insecure)`. +* Edit `/etc/exports` and add the content from this repository. * You should be able to connect to it. +## video-info.sh + +### Use case + +I bought a machine named Vero 4K+, running OSMC, to read my video library on my TV. It works fine on a number of files, but was struggling on some others, when no hardware decoder was available. I [found this thread](https://discourse.osmc.tv/t/ff-h264-dropping-and-skipping-frames/37459) that gave a first hint. I then decided to made a quick list of all the format of my files, to be able to test them. + +### Usage + +`video-info.sh` generates a CSV file with the format, the format profile, and the codec ID. It can be opened with any spreadsheet software. If it is interrupted, an option to resume the operation is available, so that it is possible to launch it multiple time on huge video libraries. + +It generates a file named `videoinfo-date_hour.csv` in the folder where the videos are located. + +By default, only the following file extensions are taken into account: `mkv`, `mp4`, and `avi`. You can easily add more by modifying the script. You can know which file extensions you have in your library by typing the following command (more [on this thread](https://stackoverflow.com/questions/1842254/how-can-i-find-all-of-the-distinct-file-extensions-in-a-folder-hierarchy)): +`find . -type f | sed -e 's/.*\.//' | sed -e 's/.*\///' | sort | uniq -c | sort -rn` + ## Utilities +* `1-update-music.sh` mounts the NAS and downloads music to the Raspberry Pi. +* `2-cover-upload.sh` extracts a cover.jpg file from songs and uploads it to the NAS. +* `3-remove-unwanted-files.sh` removes unwanted files such as `.DS_Store` and pushes those modifications to the NAS. * `equalizer.sh` allows you to modify the equalizer values of Alsamixer. -* `update-music.sh` calls the following two scripts to update your music: - * `update-music/mount-nfs.sh` mounts the NAS volume. - * `update-music/rsync-sync.sh` performs an rsync to copy the music. * `temperature.sh` displays the current Raspberry Pi temperature. -* `add-cover.sh` extracts covers from the music files to make a `cover.jpg` file. Useful for old mpd that is not able to read directly into the file. -* `video-tools/video-infos.sh` extract infos from the videos to verify if they're readable as needed. -* `usb-boot` a set of script [documented here](https://forums.raspberrypi.com/viewtopic.php?t=196778). \ No newline at end of file +* `video-infos.sh` extract infos from the videos to verify if they're readable as needed. +* `usb-boot` a set of script [documented here](https://forums.raspberrypi.com/viewtopic.php?t=196778). diff --git a/etc/exports b/etc/exports index 2265d05..dc91acd 100644 --- a/etc/exports +++ b/etc/exports @@ -8,4 +8,4 @@ # /srv/nfs4 gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check) # /srv/nfs4/homes gss/krb5i(rw,sync,no_subtree_check) # -/home/pi/video 192.168.1.0/24(rw,all_squash,insecure) +/home/pi/video 192.168.1.0/24(rw,all_squash,insecure) Vero(rw,sync,all_squash,anonuid=1000,anongid=1000) \ No newline at end of file diff --git a/etc/mpd.conf b/etc/mpd.conf index f3ab1ae..3f96350 100644 --- a/etc/mpd.conf +++ b/etc/mpd.conf @@ -11,7 +11,7 @@ # be disabled and audio files will only be accepted over ipc socket (using # file:// protocol) or streaming files over an accepted protocol. # -music_directory "/home/pi/music/punk" +music_directory "/home/pi/music/" # # This setting sets the MPD internal playlist directory. The purpose of this # directory is storage for playlists created by MPD. The server will use @@ -83,7 +83,7 @@ user "mpd" # systemd socket activation is in use. # # For network -bind_to_address "localhost" +#bind_to_address "localhost" # # And for Unix Socket #bind_to_address "/run/mpd/socket" diff --git a/utilities/1-update-music.sh b/utilities/1-update-music.sh new file mode 100644 index 0000000..4c53752 --- /dev/null +++ b/utilities/1-update-music.sh @@ -0,0 +1,3 @@ +"$HOME"/utilities/scripts/mount-nas.sh +sleep 5 +"$HOME"/utilities/scripts/download-music.sh diff --git a/utilities/2-cover-upload.sh b/utilities/2-cover-upload.sh new file mode 100644 index 0000000..0326015 --- /dev/null +++ b/utilities/2-cover-upload.sh @@ -0,0 +1,2 @@ +"$HOME"/utilities/scripts/extract-cover.sh "$HOME"/music/test +"$HOME"/utilities/scripts/upload-music.sh diff --git a/utilities/3-remove-unwanted-files.sh b/utilities/3-remove-unwanted-files.sh new file mode 100644 index 0000000..d0f1807 --- /dev/null +++ b/utilities/3-remove-unwanted-files.sh @@ -0,0 +1,2 @@ +"$HOME"/utilities/scripts/delete-files.sh "$HOME"/music/test +"$HOME"/utilities/scripts/upload-music-destructive.sh diff --git a/utilities/cover.sh b/utilities/cover.sh deleted file mode 100644 index c89c134..0000000 --- a/utilities/cover.sh +++ /dev/null @@ -1,53 +0,0 @@ -#!/bin/bash -# Recursively gets into each folder, checks if there is an audio file -# and extracts the cover with ffmpeg - -### Helpers - -# Check all extensions available -# find . -type f | perl -ne 'print $1 if m/\.([^.\/]+)$/' | sort -u - - -### Configuration - -export MUSIC_PATH="." - -### Functions - -file_exists() -{ - files=( ./*$1 ) - if [ ${#files[@]} -gt 0 ]; then - echo ${files[0]} - fi -} - -check_extensions() -{ - EXTENSIONS_SUPPORTED=("MP3" "Mp3" "mp3" "m4a") - - for i in "${EXTENSIONS_SUPPORTED[@]}" - do - file_exists "$i" - done -} - -extract_cover() -{ - if [ ! -f "cover.jpg" ] - then - shopt -s nullglob # erase not matching result - val=$(check_extensions) - #echo "file found" $val "in" $(pwd) - ffmpeg -i "$val" cover.jpg - else - echo "cover.jpg exists in " $(pwd) - fi -} - - -### Program - -find $MUSIC_PATH -type f -name ".*" -delete # remove hidden files -export -f extract_cover check_extensions file_exists -find $MUSIC_PATH \( -iname \*.MP3 -o -iname \*.Mp3 -o -iname \*.mp3 -o -iname \*.m4a \) -type f -execdir bash -c 'extract_cover "$0"' {} \; \ No newline at end of file diff --git a/utilities/scripts/delete-files.sh b/utilities/scripts/delete-files.sh new file mode 100644 index 0000000..90a6c4d --- /dev/null +++ b/utilities/scripts/delete-files.sh @@ -0,0 +1,9 @@ +export MUSIC_PATH="." + +if [ "$#" -eq 1 ]; then + export MUSIC_PATH=$1 +fi + +find $MUSIC_PATH -name '._*' -type f -delete +find $MUSIC_PATH -name '.DS_Store' -type f -delete +find $MUSIC_PATH -name '*.db' -type f -delete diff --git a/utilities/scripts/download-music.sh b/utilities/scripts/download-music.sh new file mode 100644 index 0000000..b372089 --- /dev/null +++ b/utilities/scripts/download-music.sh @@ -0,0 +1,6 @@ +sudo rsync -av --exclude="@eaDir" --delete /media/mantis-music/punk "$HOME"/music +sudo rsync -av --exclude="@eaDir" --delete /media/mantis-music/jazz "$HOME"/music +sudo rsync -av --exclude="@eaDir" --delete /media/mantis-music/chanson "$HOME"/music +sudo rsync -av --exclude="@eaDir" --delete /media/mantis-music/ost "$HOME"/music +sudo rsync -av --exclude="@eaDir" --delete /media/mantis-music/citypop "$HOME"/music +sudo rsync -av --exclude="@eaDir" --delete /media/mantis-music/electro "$HOME"/music diff --git a/utilities/scripts/extract-cover.sh b/utilities/scripts/extract-cover.sh new file mode 100644 index 0000000..14a87a2 --- /dev/null +++ b/utilities/scripts/extract-cover.sh @@ -0,0 +1,29 @@ +#!/bin/bash +# Recursively gets into each folder, checks if there is an audio file +# and extracts the cover with ffmpeg + +### Helpers + +# Check all extensions available +# find . -type f | perl -ne 'print $1 if m/\.([^.\/]+)$/' | sort -u + + +export MUSIC_PATH="." + +extract_cover() +{ + if [ ! -f "cover.jpg" ] + then + ffmpeg -hide_banner -i "$1" cover.jpg + else + echo "cover.jpg exists in " $(pwd) + fi +} + +export -f extract_cover + +if [ "$#" -eq 1 ]; then + export MUSIC_PATH=$1 +fi + +find $MUSIC_PATH \( -iname \*.MP3 -o -iname \*.Mp3 -o -iname \*.mp3 -o -iname \*.m4a \) -type f -execdir bash -c 'extract_cover "$0"' {} \; diff --git a/utilities/update-music/mount-nfs.sh b/utilities/scripts/mount-nas.sh similarity index 100% rename from utilities/update-music/mount-nfs.sh rename to utilities/scripts/mount-nas.sh diff --git a/utilities/scripts/upload-music-destructive.sh b/utilities/scripts/upload-music-destructive.sh new file mode 100644 index 0000000..9341498 --- /dev/null +++ b/utilities/scripts/upload-music-destructive.sh @@ -0,0 +1,6 @@ +sudo rsync -av --exclude="@eaDir" --delete "$HOME"/music/jazz /media/mantis-music/ +sudo rsync -av --exclude="@eaDir" --delete "$HOME"/music/punk /media/mantis-music/ +sudo rsync -av --exclude="@eaDir" --delete "$HOME"/music/chanson /media/mantis-music/ +sudo rsync -av --exclude="@eaDir" --delete "$HOME"/music/ost /media/mantis-music/ +sudo rsync -av --exclude="@eaDir" --delete "$HOME"/music/citypop /media/mantis-music/ +sudo rsync -av --exclude="@eaDir" --delete "$HOME"/music/electro /media/mantis-music/ diff --git a/utilities/scripts/upload-music.sh b/utilities/scripts/upload-music.sh new file mode 100644 index 0000000..36c944a --- /dev/null +++ b/utilities/scripts/upload-music.sh @@ -0,0 +1,6 @@ +sudo rsync -av "$HOME"/music/jazz /media/mantis-music/ +sudo rsync -av "$HOME"/music/punk /media/mantis-music/ +sudo rsync -av "$HOME"/music/chanson /media/mantis-music/ +sudo rsync -av "$HOME"/music/ost /media/mantis-music/ +sudo rsync -av "$HOME"/music/citypop /media/mantis-music/ +sudo rsync -av "$HOME"/music/electro /media/mantis-music/ diff --git a/utilities/update-music.sh b/utilities/update-music.sh deleted file mode 100644 index 0e56c87..0000000 --- a/utilities/update-music.sh +++ /dev/null @@ -1,4 +0,0 @@ -__dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -source ${__dir}/update-music/mount-nfs.sh -sleep 5 -source ${__dir}/update-music/rsync-sync.sh diff --git a/utilities/update-music/rsync-sync.sh b/utilities/update-music/rsync-sync.sh deleted file mode 100644 index 3db72d9..0000000 --- a/utilities/update-music/rsync-sync.sh +++ /dev/null @@ -1 +0,0 @@ -sudo rsync -av --exclude="@eaDir" --delete /media/mantis-music/punk /home/pi/music diff --git a/utilities/usb-boot/mbr2gpt.sh b/utilities/usb-boot/mbr2gpt similarity index 71% rename from utilities/usb-boot/mbr2gpt.sh rename to utilities/usb-boot/mbr2gpt index bb1c9e4..c43b65c 100644 --- a/utilities/usb-boot/mbr2gpt.sh +++ b/utilities/usb-boot/mbr2gpt @@ -2,11 +2,24 @@ trap '{ stty sane; echo ""; errexit "Aborted"; }' SIGINT SIGTERM -MNTPATH="/tmp/boot-gpt-mnt" +MNTPATH="/tmp/mbr2gpt-mnt" + +errexit() +{ + echo "" + echo "$1" + echo "" + if [ "${MNTED}" = "TRUE" ]; then + umount "${MNTPATH}/" &> /dev/null + fi + rm -rf "${MNTPATH}/" &> /dev/null + echo "Usage: $0 " + echo "" + exit 1 +} mntpart() { - MNTED=TRUE if [ ! -d "${MNTPATH}/" ]; then mkdir "${MNTPATH}/" if [ $? -ne 0 ]; then @@ -17,6 +30,7 @@ mntpart() if [ $? -ne 0 ]; then errexit "Unable to mount $2 partition" fi + MNTED=TRUE } umntpart() @@ -25,22 +39,8 @@ umntpart() if [ $? -ne 0 ]; then errexit "Unable to unmount partition" fi - rm -r "${MNTPATH}/" MNTED=FALSE -} - -errexit() -{ - echo "" - echo "$1" - echo "" - if [ "${MNTED}" = "TRUE" ]; then - umount "${MNTPATH}/" &> /dev/null - rm -rf "${MNTPATH}/" &> /dev/null - fi - echo "Usage: $0 /dev/sdX" - echo "" - exit 1 + rm -r "${MNTPATH}/" } MNTED=FALSE @@ -79,9 +79,13 @@ DEVICE="$1" if [ "${DEVICE}" = "" ]; then errexit "No device specified" fi -if [[ ! "${DEVICE}" =~ ^/dev/sd[a-z]$ || ! -b "${DEVICE}" ]]; then +if [[ (! "${DEVICE}" =~ ^/dev/sd[a-z]$ && ! "${DEVICE}" =~ ^/dev/nvme0n1$ && ! "${DEVICE}" =~ ^/dev/mmcblk0$) || ! -b "${DEVICE}" ]]; then errexit "Invalid DEVICE: ${DEVICE}" fi +DEVICE_P="${DEVICE}" +if [[ "${DEVICE_P}" = "/dev/nvme0n1" || "${DEVICE}" = "/dev/mmcblk0" ]];then + DEVICE_P+='p' +fi if [ $(mount | grep -c "^${DEVICE}") -ne 0 ]; then errexit "${DEVICE} is in use (mounted)" fi @@ -110,7 +114,7 @@ if [ "${PTTYPE}" = "MBR" ]; then done sgdisk -z "${DEVICE}" &> /dev/null fi -INFO="$(gdisk -l ${DEVICE})" +INFO="$(gdisk -l ${DEVICE} 2> /dev/null)" LAST=$(sed -n 's|^.*last usable sector is \(\S\+\).*|\1|p' <<< "${INFO}") START=$(sed -n 's|^\s\+2\s\+\(\S\+\).*|\1|p' <<< "${INFO}") END=$(sed -n 's|^\s\+2\s\+\S\+\s\+\(\S\+\).*|\1|p' <<< "${INFO}") @@ -174,10 +178,10 @@ if [[ "${PTTYPE}" = "MBR" || "${SHRINK}" = "TRUE" || "${EXPAND}" = "TRUE" || "${ done if [ "${SHRINK}" = "TRUE" ]; then echo "" - resize2fs -f -M "${DEVICE}2" + resize2fs -f -M "${DEVICE_P}2" fi if [[ "${SHRINK}" = "TRUE" || "${EXPAND}" = "TRUE" ]]; then - gdisk "${DEVICE}" < /dev/null + gdisk "${DEVICE}" < /dev/null d 2 n @@ -189,9 +193,9 @@ w y EOF echo "" - resize2fs -f "${DEVICE}2" + resize2fs -f "${DEVICE_P}2" fi - gdisk "${DEVICE}" < /dev/null + gdisk "${DEVICE}" < /dev/null r h 1 @@ -202,22 +206,32 @@ n w y EOF - PARTUUID_1="$(blkid ${DEVICE}1 | sed -n 's|^.*PARTUUID="\(\S\+\)".*|\1|p')" - PARTUUID_2="$(blkid ${DEVICE}2 | sed -n 's|^.*PARTUUID="\(\S\+\)".*|\1|p')" - mntpart "${DEVICE}1" "BOOT" - sed -i '/^[[:space:]]*#/!s|\(^.*rootwait\).*$|\1|' "${MNTPATH}/cmdline.txt" + PARTUUID_1="$(blkid ${DEVICE_P}1 | sed -n 's|^.*PARTUUID="\(\S\+\)".*|\1|p')" + PARTUUID_2="$(blkid ${DEVICE_P}2 | sed -n 's|^.*PARTUUID="\(\S\+\)".*|\1|p')" + mntpart "${DEVICE_P}1" "BOOT" sed -i "/^[[:space:]]*#/!s|^\(.*root=\)\S\+\(\s\+.*\)$|\1PARTUUID=${PARTUUID_2}\2|" "${MNTPATH}/cmdline.txt" + sed -i '/^[[:space:]]*#/!s| init=/usr/lib/raspi-config/init_resize\.sh||' "${MNTPATH}/cmdline.txt" umntpart - mntpart "${DEVICE}2" "ROOT" - sed -i "/^[[:space:]]*#/!s|^\S\+\(\s\+/boot\s\+.*\)$|PARTUUID=${PARTUUID_1}\1|" "${MNTPATH}/etc/fstab" + mntpart "${DEVICE_P}2" "ROOT" + sed -i "/^[[:space:]]*#/!s|^\S\+\(\s\+/boot\S*\s\+vfat\s\+.*\)$|PARTUUID=${PARTUUID_1}\1|" "${MNTPATH}/etc/fstab" sed -i "/^[[:space:]]*#/!s|^\S\+\(\s\+/\s\+.*\)$|PARTUUID=${PARTUUID_2}\1|" "${MNTPATH}/etc/fstab" + sed -i '/resize-root-fs/d' "${MNTPATH}/etc/rc.local" &> /dev/null + rm "${MNTPATH}/etc/resize-root-fs" &> /dev/null + if [ -f "${MNTPATH}/usr/lib/raspberrypi-sys-mods/firstboot" ]; then + cp "${MNTPATH}/usr/lib/raspberrypi-sys-mods/firstboot" "${MNTPATH}/usr/lib/raspberrypi-sys-mods/first-boot" + sed -i 's|firstboot|first-boot|g' "${MNTPATH}/usr/lib/raspberrypi-sys-mods/first-boot" + sed -i 's|^\(\s*whiptail --infobox \"Resizing root filesystem.*\)$| return 0\n\n\1|' "${MNTPATH}/usr/lib/raspberrypi-sys-mods/first-boot" &> /dev/null + umntpart + mntpart "${DEVICE_P}1" "BOOT" + sed -i '/^[[:space:]]*#/!s| init=/usr/lib/raspberrypi-sys-mods/firstboot| init=/usr/lib/raspberrypi-sys-mods/first-boot|' "${MNTPATH}/cmdline.txt" + fi umntpart if [ "${SDBOOT}" = "TRUE" ]; then mntpart "/dev/mmcblk0p1" "BOOT" sed -i "/^[[:space:]]*#/!s|^\(.*root=\)\S\+\(\s\+.*\)$|\1PARTUUID=${PARTUUID_2}\2|" "${MNTPATH}/cmdline.txt" umntpart - mntpart "${DEVICE}2" "ROOT" - sed -i "/^[[:space:]]*#/!s|^\S\+\(\s\+/boot\s\+.*\)$|/dev/mmcblk0p1\1|" "${MNTPATH}/etc/fstab" + mntpart "${DEVICE_P}2" "ROOT" + sed -i "/^[[:space:]]*#/!s|^\S\+\(\s\+/boot\S*\s\+vfat\s\+.*\)$|/dev/mmcblk0p1\1|" "${MNTPATH}/etc/fstab" umntpart fi if [[ "${SHRINK}" = "FALSE" && "${EXPAND}" = "FALSE" ]]; then diff --git a/utilities/usb-boot/sdc-boot.sh b/utilities/usb-boot/sdc-boot similarity index 96% rename from utilities/usb-boot/sdc-boot.sh rename to utilities/usb-boot/sdc-boot index 84e5f50..09e4130 100644 --- a/utilities/usb-boot/sdc-boot.sh +++ b/utilities/usb-boot/sdc-boot @@ -4,9 +4,22 @@ trap '{ stty sane; echo ""; errexit "Aborted"; }' SIGINT SIGTERM MNTPATH="/tmp/sdc-boot-mnt" +errexit() +{ + echo "" + echo "$1" + echo "" + if [ "${MNTED}" = "TRUE" ]; then + umount "${MNTPATH}/" &> /dev/null + fi + rm -rf "${MNTPATH}/" &> /dev/null + echo "Usage: $0 [ /dev/sdX2 | /dev/mmcblk0p2 | hhhhhhhh-02 | hhhhhhhh-hhhh-hhhh-hhhh-hhhhhhhhhhhh ]" + echo "" + exit 1 +} + mntpart() { - MNTED=TRUE if [ ! -d "${MNTPATH}/" ]; then mkdir "${MNTPATH}/" if [ $? -ne 0 ]; then @@ -17,6 +30,7 @@ mntpart() if [ $? -ne 0 ]; then errexit "Unable to mount $2 partition" fi + MNTED=TRUE } umntpart() @@ -25,22 +39,8 @@ umntpart() if [ $? -ne 0 ]; then errexit "Unable to unmount partition" fi - rm -r "${MNTPATH}/" MNTED=FALSE -} - -errexit() -{ - echo "" - echo "$1" - echo "" - if [ "${MNTED}" = "TRUE" ]; then - umount "${MNTPATH}/" &> /dev/null - rm -rf "${MNTPATH}/" &> /dev/null - fi - echo "Usage: $0 [ /dev/sdX2 | /dev/mmcblk0p2 | hhhhhhhh-02 | hhhhhhhh-hhhh-hhhh-hhhh-hhhhhhhhhhhh ]" - echo "" - exit 1 + rm -r "${MNTPATH}/" } MNTED=FALSE @@ -100,7 +100,7 @@ else umntpart if [ "${DEVICE}" != "/dev/mmcblk0p2" ]; then mntpart "${DEVICE}" "device ROOT" - sed -i "/^[[:space:]]*#/!s|^\S\+\(\s\+/boot\s\+.*\)$|/dev/mmcblk0p1\1|" "${MNTPATH}/etc/fstab" + sed -i "/^[[:space:]]*#/!s|^\S\+\(\s\+/boot\S*\s\+vfat\s\+.*\)$|/dev/mmcblk0p1\1|" "${MNTPATH}/etc/fstab" umntpart fi fi diff --git a/utilities/usb-boot/set-partuuid.sh b/utilities/usb-boot/set-partuuid similarity index 98% rename from utilities/usb-boot/set-partuuid.sh rename to utilities/usb-boot/set-partuuid index ecf9b94..15b5fca 100644 --- a/utilities/usb-boot/set-partuuid.sh +++ b/utilities/usb-boot/set-partuuid @@ -4,9 +4,22 @@ trap '{ stty sane; echo ""; errexit "Aborted"; }' SIGINT SIGTERM MNTPATH="/tmp/set-partuuid-mnt" +errexit() +{ + echo "" + echo "$1" + echo "" + if [ "${MNTED}" = "TRUE" ]; then + umount "${MNTPATH}/" &> /dev/null + fi + rm -rf "${MNTPATH}/" &> /dev/null + echo "Usage: $0 device [ hhhhhhhh-02 | hhhhhhhh-hhhh-hhhh-hhhh-hhhhhhhhhhhh | random ]" + echo "" + exit 1 +} + mntpart() { - MNTED=TRUE if [ ! -d "${MNTPATH}/" ]; then mkdir "${MNTPATH}/" if [ $? -ne 0 ]; then @@ -17,6 +30,7 @@ mntpart() if [ $? -ne 0 ]; then errexit "Unable to mount $2 partition" fi + MNTED=TRUE } umntpart() @@ -25,22 +39,8 @@ umntpart() if [ $? -ne 0 ]; then errexit "Unable to unmount partition" fi - rm -r "${MNTPATH}/" MNTED=FALSE -} - -errexit() -{ - echo "" - echo "$1" - echo "" - if [ "${MNTED}" = "TRUE" ]; then - umount "${MNTPATH}/" &> /dev/null - rm -rf "${MNTPATH}/" &> /dev/null - fi - echo "Usage: $0 device [ hhhhhhhh-02 | hhhhhhhh-hhhh-hhhh-hhhh-hhhhhhhhhhhh | random ]" - echo "" - exit 1 + rm -r "${MNTPATH}/" } MNTED=FALSE @@ -149,7 +149,7 @@ EOF PARTUUID_1="$(blkid "${DEVICE_P}1" | sed -n 's|^.*PARTUUID="\(\S\+\)".*|\1|p')" PARTUUID_2="$(blkid "${DEVICE_P}2" | sed -n 's|^.*PARTUUID="\(\S\+\)".*|\1|p')" mntpart "${DEVICE_P}2" "ROOT" - sed -i "/^[[:space:]]*#/!s|^\(PARTUUID=\)${ORIGUUID_1}\(\s\+/boot\s\+.*\)$|\1${PARTUUID_1}\2|" "${MNTPATH}/etc/fstab" + sed -i "/^[[:space:]]*#/!s|^\(PARTUUID=\)${ORIGUUID_1}\(\s\+/boot\S*\s\+vfat\s\+.*\)$|\1${PARTUUID_1}\2|" "${MNTPATH}/etc/fstab" sed -i "/^[[:space:]]*#/!s|^\(PARTUUID=\)${ORIGUUID_2}\(\s\+/\s\+.*\)$|\1${PARTUUID_2}\2|" "${MNTPATH}/etc/fstab" umntpart mntpart "${DEVICE_P}1" "BOOT" diff --git a/utilities/usb-boot/usb-boot.sh b/utilities/usb-boot/usb-boot similarity index 69% rename from utilities/usb-boot/usb-boot.sh rename to utilities/usb-boot/usb-boot index b01da6b..150599e 100644 --- a/utilities/usb-boot/usb-boot.sh +++ b/utilities/usb-boot/usb-boot @@ -4,9 +4,21 @@ trap '{ stty sane; echo ""; errexit "Aborted"; }' SIGINT SIGTERM MNTPATH="/tmp/usb-boot-mnt" +errexit() +{ + echo "" + echo "$1" + echo "" + if [ "${MNTED}" = "TRUE" ]; then + umount "${BOOTMNT}/" &> /dev/null + umount "${MNTPATH}/" &> /dev/null + fi + rm -rf "${MNTPATH}/" &> /dev/null + exit 1 +} + mntdev() { - MNTED=TRUE if [ ! -d "${MNTPATH}/" ]; then mkdir "${MNTPATH}/" if [ $? -ne 0 ]; then @@ -17,13 +29,14 @@ mntdev() if [ $? -ne 0 ]; then errexit "Unable to mount ROOT partition" fi - if [ ! -d "${MNTPATH}/boot/" ]; then - mkdir -p "${MNTPATH}/boot/" + MNTED=TRUE + if [ ! -d "${BOOTMNT}/" ]; then + mkdir -p "${BOOTMNT}/" if [ $? -ne 0 ]; then errexit "Unable to make BOOT partition mount point" fi fi - mount "${1}1" "${MNTPATH}/boot/" + mount "${1}1" "${BOOTMNT}/" if [ $? -ne 0 ]; then errexit "Unable to mount BOOT partition" fi @@ -31,7 +44,7 @@ mntdev() umntdev() { - umount "${MNTPATH}/boot/" + umount "${BOOTMNT}/" if [ $? -ne 0 ]; then errexit "Unable to unmount BOOT partition" fi @@ -39,21 +52,8 @@ umntdev() if [ $? -ne 0 ]; then errexit "Unable to unmount ROOT partition" fi - rm -r "${MNTPATH}/" MNTED=FALSE -} - -errexit() -{ - echo "" - echo "$1" - echo "" - if [ "${MNTED}" = "TRUE" ]; then - umount "${MNTPATH}/boot/" &> /dev/null - umount "${MNTPATH}/" &> /dev/null - rm -rf "${MNTPATH}/" &> /dev/null - fi - exit 1 + rm -r "${MNTPATH}/" } MNTED=FALSE @@ -92,9 +92,15 @@ rsync --version &> /dev/null if [ $? -ne 0 ]; then errexit "rsync not installed (run: apt-get install rsync)" fi +BOOTMNT="${MNTPATH}$(sed -n 's|^\S\+\s\+\(/boot\S*\)\s\+.*$|\1|p' /etc/fstab)" +if [ "${BOOTMNT}" = "${MNTPATH}/boot/firmware" ]; then + BOOTSIZE=512 +else + BOOTSIZE=256 +fi ROOT_PART="$(mount | sed -n 's|^\(/dev/.*\) on / .*|\1|p')" ROOT_DEV="$(sed 's/[0-9]\+$//' <<< "${ROOT_PART}")" -if [ "${ROOT_DEV}" = "/dev/mmcblk0p" ]; then +if [[ "${ROOT_DEV}" = "/dev/mmcblk0p" || "${ROOT_DEV}" = "/dev/nvme0n1p" ]]; then ROOT_DEV="${ROOT_DEV:0:(${#ROOT_DEV} - 1)}" fi ROOT_TYPE="$(blkid "${ROOT_PART}" | sed -n 's|^.*TYPE="\(\S\+\)".*|\1|p')" @@ -103,17 +109,27 @@ if [ -b /dev/mmcblk0 ]; then else USESDC=FALSE fi -if [[ $(cat /proc/cpuinfo | grep -c "^Revision\s\+:\s\+[abcd]0311[1245]$") -ne 0 || $(cat /proc/cpuinfo | grep -c "^Revision\s\+:\s\+c03130$") -ne 0 ]]; then - RPI_4=TRUE +if [ -f "/sys/firmware/devicetree/base/system/linux,revision" ]; then + BDINFO="$(od -v -An -t x1 /sys/firmware/devicetree/base/system/linux,revision | tr -d ' \n')" +elif grep -q Revision /proc/cpuinfo; then + BDINFO="$(sed -n '/^Revision/s/^.*: \(.*\)/\1/p' < /proc/cpuinfo)" +elif command -v vcgencmd > /dev/null; then + BDINFO="$(vcgencmd otp_dump | grep '30:' | sed 's/.*://')" else - RPI_4=FALSE + errexit "Raspberry Pi board info not found" +fi +BDCHIP=$(((0x${BDINFO} >> 12) & 15)) +if [[ ${BDCHIP} = 3 || ${BDCHIP} = 4 ]]; then + RPI_45=TRUE +else + RPI_45=FALSE fi if [ "$(vcgencmd otp_dump | grep 17:)" = "17:3020000a" ]; then RPI_3=TRUE else RPI_3=FALSE fi -if [[ "${USESDC}" = "TRUE" && "${RPI_4}" = "TRUE" ]]; then +if [[ "${USESDC}" = "TRUE" && "${RPI_45}" = "TRUE" ]]; then whiptail --backtitle "USB Boot" --title "USB Boot Method" --yesno --defaultno "\nUse SD card to boot the USB device?" 12 40 YESNO=$? if [ ${YESNO} -eq 255 ]; then @@ -129,10 +145,10 @@ elif [[ "${USESDC}" = "TRUE" && "${RPI_3}" = "TRUE" ]]; then elif [ ${YESNO} -ne 0 ]; then USESDC=FALSE fi -elif [[ "${USESDC}" = "FALSE" && "${RPI_4}" = "FALSE" && "${RPI_3}" = "FALSE" ]]; then - errexit "Not a Raspberry Pi 4, Raspberry Pi 3B+, or Raspberry Pi 3B with OTP set, and no SD card present or used" +elif [[ "${USESDC}" = "FALSE" && "${RPI_45}" = "FALSE" && "${RPI_3}" = "FALSE" ]]; then + errexit "Not a Raspberry Pi 5, Raspberry Pi 4, Raspberry Pi 3B+, or Raspberry Pi 3B with OTP set, and no SD card present or used" fi -USBDEVS=($(ls -l /dev/sd? 2> /dev/null | sed -n 's|^.*\(/dev/.*\)|\1|p')) +USBDEVS=($(ls -l /dev/sd? /dev/nvme0n1 2> /dev/null | sed -n 's|^.*\(/dev/.*\)|\1|p')) if [ ${#USBDEVS[@]} -eq 0 ]; then errexit "No available USB mass storage devices found" fi @@ -144,8 +160,12 @@ USB_DEST="$(whiptail --backtitle "USB Boot" --title "USB Mass Storage Devices" - if [[ $? -ne 0 || "${USB_DEST}" = "" ]]; then errexit "Aborted" fi -USB_BOOT="${USB_DEST}1" -USB_ROOT="${USB_DEST}2" +USB_DEST_P=${USB_DEST} +if [ "${USB_DEST_P}" = "/dev/nvme0n1" ]; then + USB_DEST_P+='p' +fi +USB_BOOT="${USB_DEST_P}1" +USB_ROOT="${USB_DEST_P}2" if [ "${ROOT_DEV}" != "${USB_DEST}" ]; then whiptail --backtitle "USB Boot" --title "Replicate BOOT/ROOT Contents" --yesno "\nReplicate BOOT/ROOT contents from ${ROOT_DEV} to ${USB_DEST}?" 12 64 YESNO=$? @@ -168,7 +188,7 @@ if [ "${ROOT_DEV}" != "${USB_DEST}" ]; then if [[ $? -ne 0 || "${PTTYPE}" = "" ]]; then errexit "Aborted" fi - whiptail --backtitle "USB Boot" --title "WARNING" --yesno "\nWARNING\n\nAll existing data on USB device ${USB_DEST} will be destroyed!\n\nDo you wish to continue?" 14 64 + whiptail --backtitle "USB Boot" --title "WARNING" --yesno "\nWARNING\n\nAll existing data on USB device ${USB_DEST} will be destroyed!\n\nDo you wish to continue?" 14 68 YESNO=$? if [ ${YESNO} -ne 0 ]; then errexit "Aborted" @@ -182,18 +202,18 @@ if [ "${ROOT_DEV}" != "${USB_DEST}" ]; then partprobe echo "label: dos" | sfdisk "${USB_DEST}" > /dev/null sfdisk "${USB_DEST}" < /dev/null -,256MiB,c +,${BOOTSIZE}MiB,c ,+,83 EOF else sgdisk -Z "${USB_DEST}" &> /dev/null - sgdisk -n 1:0:+256M "${USB_DEST}" &> /dev/null + sgdisk -n 1:0:+${BOOTSIZE}M "${USB_DEST}" &> /dev/null sgdisk -t 1:0700 "${USB_DEST}" > /dev/null sgdisk -n 2:0:0 "${USB_DEST}" &> /dev/null sgdisk -t 2:8300 "${USB_DEST}" > /dev/null fi partprobe - mkfs.vfat -F 32 "${USB_BOOT}" > /dev/null + mkfs.vfat -F 32 -n boot -s 4 "${USB_BOOT}" &> /dev/null if [ $? -ne 0 ]; then errexit "Unable to create BOOT filesystem" fi @@ -204,12 +224,12 @@ EOF if [ "${ROOT_TYPE}" = "f2fs" ]; then mkfs.f2fs -f "${USB_ROOT}" > /dev/null else - mkfs.ext4 -F -q -b 4096 "${USB_ROOT}" > /dev/null + mkfs.ext4 -b 4096 -F -L rootfs -q "${USB_ROOT}" > /dev/null fi if [ $? -ne 0 ]; then errexit "Unable to create ROOT filesystem" fi - mntdev "${USB_DEST}" + mntdev "${USB_DEST_P}" rsync -aDH --partial --numeric-ids --delete --force --exclude "${MNTPATH}" --exclude '/dev' --exclude '/lost+found' --exclude '/media' --exclude '/mnt' \ --exclude '/proc' --exclude '/run' --exclude '/sys' --exclude '/tmp' --exclude '/etc/udev/rules.d/70-persistent-net.rules' \ --exclude '/var/lib/asterisk/astdb.sqlite3-journal' "${OPTIONS[@]}" / "${MNTPATH}/" @@ -228,7 +248,7 @@ EOF fi if [ "${USESDC}" = "TRUE" ]; then if [ -b /dev/mmcblk0 ]; then - while [ "$(blkid /dev/mmcblk0p2 | sed -n 's|^.*PARTUUID="\(\S\+\)".*|\1|p')" = "$(blkid ${USB_DEST}2 | sed -n 's|^.*PARTUUID="\(\S\+\)".*|\1|p')" ] + while [ "$(blkid /dev/mmcblk0p2 | sed -n 's|^.*PARTUUID="\(\S\+\)".*|\1|p')" = "$(blkid ${USB_DEST_P}2 | sed -n 's|^.*PARTUUID="\(\S\+\)".*|\1|p')" ] do echo "" echo "WARNING : SD card (/dev/mmcblk0) and USB device (${USB_DEST}) have the same PARTUUID : SD card will boot instead of USB device" @@ -283,97 +303,28 @@ PARTUUID_2="$(blkid "${USB_ROOT}" | sed -n 's|^.*PARTUUID="\(\S\+\)".*|\1|p')" if [ "${USESDC}" = "TRUE" ]; then if [ -b /dev/mmcblk0 ]; then mntdev "/dev/mmcblk0p" - sed -i "/^[[:space:]]*#/!s|^\(.*root=\)\S\+\(\s\+.*\)$|\1PARTUUID=${PARTUUID_2}\2|" "${MNTPATH}/boot/cmdline.txt" + sed -i "/^[[:space:]]*#/!s|^\(.*root=\)\S\+\(\s\+.*\)$|\1PARTUUID=${PARTUUID_2}\2|" "${BOOTMNT}/cmdline.txt" umntdev else errexit "No SD card present" fi fi -mntdev "${USB_DEST}" -sed -i "/^[[:space:]]*#/!s|^\(.*root=\)\S\+\(\s\+.*\)$|\1PARTUUID=${PARTUUID_2}\2|" "${MNTPATH}/boot/cmdline.txt" +mntdev "${USB_DEST_P}" +sed -i "/^[[:space:]]*#/!s|^\(.*root=\)\S\+\(\s\+.*\)$|\1PARTUUID=${PARTUUID_2}\2|" "${BOOTMNT}/cmdline.txt" sed -i "/^[[:space:]]*#/!s|^\S\+\(\s\+/\s\+.*\)$|PARTUUID=${PARTUUID_2}\1|" "${MNTPATH}/etc/fstab" if [ "${USESDC}" = "TRUE" ]; then - sed -i "/^[[:space:]]*#/!s|^\S\+\(\s\+/boot\s\+.*\)$|/dev/mmcblk0p1\1|" "${MNTPATH}/etc/fstab" - if [ $(grep -v '^[[:space:]]*#' "${MNTPATH}/etc/rc.local" | grep -c 'usb-boot-init') -eq 0 ]; then - cat <<\EOF1 > "${MNTPATH}/etc/usb-boot-init" -#!/bin/bash - -MNTBOOT="/tmp/usb-boot-init-mnt" - -REBOOT=FALSE -ROOT_PART="$(mount | sed -n 's|^\(/dev/.*\) on / .*|\1|p')" -ROOT_DEV="$(sed 's/[0-9]\+$//' <<< "${ROOT_PART}")" -if [ "${ROOT_DEV}" = "/dev/mmcblk0p" ]; then - ROOT_DEV="${ROOT_DEV:0:(${#ROOT_DEV} - 1)}" -fi -mkdir -p "${MNTBOOT}/" -mount "${ROOT_DEV}1" "${MNTBOOT}/" -if [ "$(find "${MNTBOOT}/" -type f -iregex "${MNTBOOT}/ssh\(\.txt\)?" -print -delete)" != "" ]; then - update-rc.d ssh enable && invoke-rc.d ssh start -fi -if [ -f "${MNTBOOT}/wpa_supplicant.conf" ]; then - mv "${MNTBOOT}/wpa_supplicant.conf" /etc/wpa_supplicant/wpa_supplicant.conf - REBOOT=TRUE -fi -if [[ $(grep -v '^[[:space:]]*#' "${MNTBOOT}/cmdline.txt" | grep -c 'init=/usr/lib/raspi-config/init_resize.sh') -ne 0 ]]; then - sed -i '/^[[:space:]]*#/!s| init=/usr/lib/raspi-config/init_resize\.sh||' "${MNTBOOT}/cmdline.txt" - START2=$(sfdisk -d "${ROOT_DEV}" | sed -n "s|^${ROOT_PART}\s\+:.*start=\s*\([0-9]\+\).*$|\1|p") - PTTYPE="$(blkid "${ROOT_DEV}" | sed -n 's|^.*PTTYPE="\(\S\+\)".*|\1|p')" - if [ "${PTTYPE}" = "dos" ]; then - sfdisk --delete "${ROOT_DEV}" 2 > /dev/null - echo "${START2},+" | sfdisk --no-reread --no-tell-kernel -N2 "${ROOT_DEV}" &> /dev/null - else - PARTUUID="$(blkid "${ROOT_PART}" | sed -n 's|^.*PARTUUID="\(\S\+\)".*|\1|p')" - sgdisk -d 2 "${ROOT_DEV}" > /dev/null - sgdisk -n 2:${START2}:0 "${ROOT_DEV}" > /dev/null - sgdisk -u 2:${PARTUUID} "${ROOT_DEV}" > /dev/null - fi - cat <<\EOF2 > /etc/init.d/resize_root_fs -#!/bin/sh -### BEGIN INIT INFO -# Provides: resize_root_fs -# Required-Start: -# Required-Stop: -# Default-Start: 3 -# Default-Stop: -# Short-Description: Resize root filesystem -# Description: -### END INIT INFO - -case "$1" in - start) - resize2fs "$(mount | sed -n 's|^\(/dev/.*\) on / .*|\1|p')" - update-rc.d resize_root_fs remove - rm /etc/init.d/resize_root_fs - ;; - *) - exit 3 - ;; -esac -EOF2 - chmod +x /etc/init.d/resize_root_fs - update-rc.d resize_root_fs defaults - REBOOT=TRUE -fi -umount "${MNTBOOT}/" -rm -r "${MNTBOOT}/" -sed -i '/usb-boot-init/d' /etc/rc.local -rm /etc/usb-boot-init -if [ "${REBOOT}" = "TRUE" ]; then - shutdown --no-wall -r now -fi -EOF1 - chmod +x "${MNTPATH}/etc/usb-boot-init" - sed -i 's|^exit 0$|/etc/usb-boot-init\nexit 0|' "${MNTPATH}/etc/rc.local" - fi + sed -i "/^[[:space:]]*#/!s|^\S\+\(\s\+/boot\S*\s\+vfat\s\+.*\)$|/dev/mmcblk0p1\1|" "${MNTPATH}/etc/fstab" else - sed -i "/^[[:space:]]*#/!s|^\S\+\(\s\+/boot\s\+.*\)$|PARTUUID=${PARTUUID_1}\1|" "${MNTPATH}/etc/fstab" + sed -i "/^[[:space:]]*#/!s|^\S\+\(\s\+/boot\S*\s\+vfat\s\+.*\)$|PARTUUID=${PARTUUID_1}\1|" "${MNTPATH}/etc/fstab" fi umntdev DEV_LIST=() if [ -b /dev/mmcblk0 ]; then DEV_LIST+=/dev/mmcblk0p2 fi +if [ -b /dev/nvme0n1 ]; then + DEV_LIST+=/dev/nvme0n1p2 +fi DEV_LIST+=($(ls -l /dev/sd?2 2> /dev/null | sed -n 's|^.*\(/dev/.*\)|\1|p')) if [ ${#DEV_LIST[@]} -gt 1 ]; then for i in ${!DEV_LIST[@]}; do @@ -381,7 +332,7 @@ if [ ${#DEV_LIST[@]} -gt 1 ]; then j=$((i + 1)) while [ ${j} -lt ${#DEV_LIST[@]} ]; do if [ "$(blkid "${DEV_LIST[i]}" | sed -n 's|^.*PARTUUID="\(\S\+\)".*|\1|p')" = "$(blkid "${DEV_LIST[j]}" | sed -n 's|^.*PARTUUID="\(\S\+\)".*|\1|p')" ]; then - if [[ "${DEV_LIST[i]}" != "/dev/mmcblk0p2" || "${DEV_LIST[j]}" != "${USB_DEST}2" ]]; then + if [[ "${DEV_LIST[i]}" != "/dev/mmcblk0p2" || "${DEV_LIST[j]}" != "${USB_DEST_P}2" ]]; then echo "" echo "WARNING : ${DEV_LIST[i]} and ${DEV_LIST[j]} have the same PARTUUID : $(blkid "${DEV_LIST[i]}" | sed -n 's|^.*PARTUUID="\(\S\+\)".*|\1|p')" fi diff --git a/utilities/usb-boot/usb-boot.txt b/utilities/usb-boot/usb-boot.txt index b4309ed..39b5794 100644 --- a/utilities/usb-boot/usb-boot.txt +++ b/utilities/usb-boot/usb-boot.txt @@ -1,16 +1,14 @@ -Running Raspbian on USB Devices : Made Easy +Running Raspberry Pi OS on USB Devices : Made Easy -A recurring topic of discussion is how to configure and reliably run Raspbian on a USB flash drive, USB hard drive, or USB SSD instead of an SD card. +A recurring topic of discussion is how to configure and reliably run Raspberry Pi OS on a USB flash drive, USB hard drive, or USB SSD instead of an SD card. -A Raspberry Pi 3B+ has a native USB boot mode (this mode has to be manually enabled by setting an OTP bit on a Raspberry Pi 3B). This native USB boot mode has serious compatibility issues. A bootcode.bin file is available for older Raspberry Pi models. Unfortunately, both of these approaches have serious limitations and once working, can easily be broken by simply plugging in an additional USB storage device. +A Raspberry Pi 4 or Raspberry Pi 5 has a native USB boot mode that is reliable and should be used. -The easiest and most reliable way to run Raspbian on a USB device with any Raspberry Pi is to leave an SD card containing Raspbian in place, but use it only for starting Raspbian that is residing on a USB device. While setting up such a configuration is not rocket science, it can be confusing to a newcomer or someone unfamiliar with Linux internals. In an effort to simplify the task, I've created the attached script named 'usb-boot' to automate the process. +A Raspberry Pi 3B+ has a native USB boot mode (this mode has to be manually enabled by setting an OTP bit on a Raspberry Pi 3B). This native USB boot mode has serious compatibility issues. A bootcode.bin file is available for older Raspberry Pi models. Unfortunately, these have serious limitations and once working, can easily become broken. The easiest and most reliable way to run Raspberry Pi OS on a USB device with any Raspberry Pi prior to the model 4 or 5 is to leave an SD card containing Raspberry Pi OS in place, but use it only for starting the Raspberry Pi OS that is residing on a USB device. -If usb-boot is running on a Raspberry Pi 4, usb-boot first prompts: 'Use SD card to boot the USB device?' +If usb-boot is running on a Raspberry Pi 4 or Raspberry Pi 5, usb-boot first prompts: 'Use SD card to boot the USB device?' -If 'No' is selected, the SD card will not be altered and the direct USB boot capability of the Raspberry Pi 4 will be used. - -[NOTE: The direct USB boot capability of the Raspberry Pi 4 requires an updated bootloader and firmware: https://www.raspberrypi.org/documentation/hardware/raspberrypi/bcm2711_bootloader_config.md] +If 'No' is selected, the SD card will not be altered and the direct USB boot capability of the Raspberry Pi 4 or Raspberry Pi 5 will be used. If usb-boot is running on a Raspberry Pi 3B+ or a Raspberry Pi 3B with its OTP bit set, usb-boot first prompts: 'Use SD card to boot the USB device (recommended)?' @@ -24,9 +22,9 @@ usb-boot will then prompt: 'Replicate BOOT/ROOT contents from /dev/mmcblk0 to /d /dev/mmcblk0 is the SD card and /dev/sdX is the USB device. -Select 'No' if the USB device already has Raspbian on it and you wish to use it (nothing will be copied). +Select 'No' if the USB device already has Raspberry Pi OS on it and you wish to use it (nothing will be copied). -Select 'Yes' if you want to copy the Raspbian on your SD card to the USB device (everything will be copied). +Select 'Yes' if you want to copy the Raspberry Pi OS on your SD card to the USB device (everything will be copied). If you select 'Yes', usb-boot will then prompt: 'Select the partition table type to use (MBR = 2TB Maximum)' @@ -36,7 +34,7 @@ If you select 'Yes', the copy will begin. The time required for this process wi usb-boot will then complete the configuration process and warn you of any potential conflicts it detects. -When usb-boot has finished, you should be able to reboot and be running Raspbian on the USB device (first power off and remove the SD card if not using the SD card to boot the USB device). +When usb-boot has finished, you should be able to reboot and be running Raspberry Pi OS on the USB device (first power off and remove the SD card if not using the SD card to boot the USB device). ===== @@ -59,10 +57,10 @@ If no device is specified, the currently selected boot device will be displayed. GPT partition tables are necessary for devices whose size is over 2TB. mbr2gpt converts an MBR partition table on a USB device to a GPT partition table, as well as optionally expanding the ROOT partition and enabling booting via an SD card. + mbr2gpt converts any size USB device, including SD cards placed in a USB adapter. !!! DO NOT PROCEED UNLESS YOU HAVE A RELIABLE BACKUP OF THE DEVICE BEING CONVERTED !!! -!!! INITIAL TESTING SHOULD BE PERFORMED ON A USB DEVICE CONTAINING EXPENDABLE DATA !!! Usage syntax is: diff --git a/utilities/video-tools/video-info.sh b/utilities/video-info.sh similarity index 83% rename from utilities/video-tools/video-info.sh rename to utilities/video-info.sh index f1b52c9..afb06b8 100644 --- a/utilities/video-tools/video-info.sh +++ b/utilities/video-info.sh @@ -5,24 +5,31 @@ normal=$(tput sgr0) # $1 = video file / $2 = result file get_mediainfo() { - echo "Processing $1..." - echo -ne "\"$1\", " >> $2 - mediainfo --Inform="Video;%Format%, %Format_Profile%, %CodecID%" "$1" >> $2 + echo "Processing " "$1" "..." + echo -ne "\" "$1" \", " >> "$2" + mediainfo --Inform="Video;%Format%, %Format_Profile%, %CodecID%" "$1" >> "$2" } gather_data() { echo "Gathering data..." + + echo "Test path => " "$1" RESULT_PATH=./$1/videoinfo-"$CURRENT_DATE".csv PWD=$(pwd $1)/$1 - echo $PWD >> $RESULT_PATH - echo 'File, Format, Format Profile, Codec ID' >> $RESULT_PATH - find $1 -type f \( -iname \*.mkv -o -iname \*.mp4 -o -iname \*.avi \) | while read file; + echo "$PWD" + echo "$RESULT_PATH" + + echo "$PWD" >> "$RESULT_PATH" + echo 'File, Format, Format Profile, Codec ID' >> "$RESULT_PATH" + find "$1" -type f \( -iname \*.mkv -o -iname \*.mp4 -o -iname \*.avi \) | while read file; do - get_mediainfo "$(pwd $1)/$file" $RESULT_PATH + get_mediainfo "$(pwd $1)/$file" "$RESULT_PATH" done echo "Finished! Result file is located at ${bold}$RESULT_PATH${normal}." + + exit } gather_data_continue() { @@ -82,9 +89,10 @@ elif [[ $1 == "-c" ]] || [[ $1 == "--continue" ]]; then continue_gather $2 else if [[ -d $1 ]]; then - gather_data $1 + echo "Gather data" + gather_data "$1" else echo -e "Directory ${bold}$1${normal} does not exist. Showing usage...\n" show_usage fi -fi \ No newline at end of file +fi diff --git a/utilities/video-tools/README.md b/utilities/video-tools/README.md deleted file mode 100644 index 5626966..0000000 --- a/utilities/video-tools/README.md +++ /dev/null @@ -1,24 +0,0 @@ -# video-tools - -## Use case - -I bought a machine named Vero 4K+, running OSMC, to read my video library on my TV. It works fine on a number of files, but was struggling on some others, when no hardware decoder was available. I [found this thread](https://discourse.osmc.tv/t/ff-h264-dropping-and-skipping-frames/37459) that gave a first hint. I then decided to made a quick list of all the format of my files, to be able to test them. That's how this little project started. - -## Set up - -* On a Linux machine with apt: `sudo apt install -y ffmpeg mediainfo` -* If you are on Windows, you can: - * [install WSL](https://docs.microsoft.com/en-us/windows/wsl/install-win10) - * install Debian/Ubuntu/etc and type in: `sudo apt install -y ffmpeg mediainfo` - * If you access a networked drive, you can access it like this for example: - * `sudo mkdir /mnt/multimedia` - * `sudo mount -t drvfs //Vero/Multimedia /mnt/multimedia/` (this need to be done at each reboot) - -## video-info.sh - -video-info.sh generates a CSV file with the format, the format profile, and the codec ID. It can be opened with any spreadsheet software. If it is interrupted, an option to resume the operation is available, so that it is possible to launch it multiple time on huge video libraries. - -It generates a file named `videoinfo-date_hour.csv` in the folder where the videos are located. - -By default, only the following file extensions are taken into account: `mkv`, `mp4`, and `avi`. You can easily add more by modifying the script. You can know which file extensions you have in your library by typing the following command (more [on this thread](https://stackoverflow.com/questions/1842254/how-can-i-find-all-of-the-distinct-file-extensions-in-a-folder-hierarchy)): -`find . -type f | sed -e 's/.*\.//' | sed -e 's/.*\///' | sort | uniq -c | sort -rn` \ No newline at end of file