Tag: diy

  • How I Saved My MacBook Pro From Bad RAM

    How I Saved My MacBook Pro From Bad RAM

    Oh boy, how did we end up here?

    When the first macOS Sequoia public beta came out, I went ahead and installed it on my 2019 Intel MacBook Pro. I’ve never had a problem with public betas, and usually try them out.

    Well, this time something went wrong. I don’t remember the details, but the install crashed, froze, or borked somehow near the end. I restarted and it seemed to finish.

    … but we wouldn’t be here if that were the end of the story…

    Something felt off, and whenever I rebooted my MacBook Pro, anything I did reverted itself to back to the end of the public beta install. I was stuck in some sort of disk snapshot mode. I tried repairing the disk, I tried rebooting into single user mode and fixing things, all sorts of stuff I tried to think of, stuff I googled, and stuff I dangerously got from ChatGPT. Nothing would fix it to where I could permanently modify my drive anymore.

    So I did what anyone would do–I backed up my important files and completely reformatted the drive and did a fresh install once the final version of Sequoia came out.

    And that worked, and things were fine. Except for some random crashes every now and then. Sometimes a Firefox tab would crash. Sometimes the machine would not come out of sleep. Sometimes Messages would not sync from iCloud. Sometimes entire programs would crash.

    I had just assumed it was the perils of the new macOS on older hardware, and I kept going forward… 15.1 came out, and problems persisted… 15.2…. 15.3… well, by now any bugs should be fixed! Something is wrong. I of course understand that these random crashes feel a lot like bad memory, but I was in denial. The MacBook had soldered RAM and there was nothing I could 😭

    I first went with Memtester as a quick test, just to confirm that my RAM is fine:

    $ sudo memtester 16G 2
    Password:
    memtester version 4.7.0 (64-bit)
    Copyright (C) 2001-2024 Charles Cazabon.
    Licensed under the GNU General Public License version 2 (only).
    
    pagesize is 4096
    pagesizemask is 0xfffffffffffff000
    want 16384MB (17179869184 bytes)
    got  16384MB (17179869184 bytes), trying mlock ...locked.
    Loop 1/2:
      Stuck Address       : testing   1FAILURE: possible bad address line at offset 0x0000000254c27038.
    Skipping to next test...
      Random Value        : FAILURE: 0x06bf7b9befebc9e5 != 0x0ebf7b9befebc9e5 at offset 0x0000000054c27840.
    FAILURE: 0x2106487df0007127 != 0x2906487df0007127 at offset 0x0000000054c27030.
    FAILURE: 0x2106487df0007127 != 0x2906487df0007127 at offset 0x0000000054c27038.
    FAILURE: 0x2106487df0007127 != 0x2906487df0007127 at offset 0x0000000054c28238.
      Compare XOR         :   Compare SUB         : ok
    FAILURE: 0x0400ed73c8282b4d != 0x0c00ed73c8282b4d at offset 0x0000000054c27038.
    FAILURE: 0x0400ed73c8282b4d != 0x0c00ed73c8282b4d at offset 0x0000000054c28238.
      Compare MUL         : FAILURE: 0x0000000000000002 != 0x0800000000000002 at offset 0x0000000054c27030.
    FAILURE: 0x0000000000000002 != 0x0800000000000002 at offset 0x0000000054c27038.
    FAILURE: 0x0000000000000002 != 0x0800000000000002 at offset 0x0000000054c28238.
      Compare DIV         :   Compare OR          : ok
      Compare AND         : ok
      Sequential Increment: ok
      Solid Bits          : testing   0FAILURE: 0x0000000000000000 != 0x0800000000000000 at offset 0x0000000054c27038.
    FAILURE: 0x0000000000000000 != 0x0800000000000000 at offset 0x0000000054c28238.
      Block Sequential    : testing   0FAILURE: 0x0000000000000000 != 0x0800000000000000 at offset 0x0000000054c27030.
    FAILURE: 0x0000000000000000 != 0x0800000000000000 at offset 0x0000000054c27038.
    FAILURE: 0x0000000000000000 != 0x0800000000000000 at offset 0x0000000054c28238.
      Checkerboard        : testing   1FAILURE: 0x5555555555555555 != 0x5d55555555555555 at offset 0x0000000054c27038.
    FAILURE: 0x5555555555555555 != 0x5d55555555555555 at offset 0x0000000054c28238.
      Bit Spread          : testing  57FAILURE: 0xf5ffffffffffffff != 0xfdffffffffffffff at offset 0x0000000054c27038.
      Bit Flip            : testing   0FAILURE: 0x0000000000000001 != 0x0800000000000001 at offset 0x0000000054c27038.
    FAILURE: 0x0000000000000001 != 0x0800000000000001 at offset 0x0000000054c28238.
      Walking Ones        : setting   4Code language: PHP (php)

    I was nervous! There were failures! But I also knew that testing live in the OS isn’t perfect, so maybe there was a chance! I whipped up a flash drive with Memtest86 to prove that my memory was still good. Phew.

    Yeah… about that…

    I had a serious problem! This machine is still perfectly fine. I had to make a choice:

    1. I could try to repair it myself. I’ve never done any BGA work myself, but I’ve seen plenty of it done and honestly how hard could it be? *nervous chuckle*
    2. I could pay to get this fixed. That’s the safest solution.
    3. I could install Linux and use badram to just ignore the problem. The problem with this is that I’d lose all of the MacOS magic with the ecosystem.

    Alright, so let’s go with the safest option first. I fired off a quote request to Rossman Repair Group, who seem to be the best at this stuff.

    *sad trombone noises*

    Alright, so the choices are clear. I can either start hacking away at my hardware or try some software based solutions. I knew that Linux had a nice badram option with Grub, so I started searching around to see if there was any way at all to get that working for macOS.

    I searched and continued to come up with nothing. There was no way that I could find to get badram to work with macOS. Bummer.

    …there’s a glimmer of hope though…

    In my many searches, I ran across this project:

    Could this be what I need? Basically a badram type functionality for macOS? Sweet! Alright, let’s try this out.

    So I can’t just magically install this software. It’s part of the EFI boot loader, so we need to install it there.

    BUT WAIT THERE’S MORE. From what I was able to tell, I needed an EFI shell because macOS doesn’t have a built-in way to blacklist bad RAM addresses at boot like Linux’s badram does. So the plan was simple:

    1. Install an EFI shell that would let me run the RAM-disabling utility before macOS boots.
    2. Set up rEFInd as a boot manager to automate the process.
    3. Write an EFI shell script to block the bad RAM every time the system starts.


    First, I grabbed the TianoCore UEFI Shell (aka Shell_Full.efi) from the official TianoCore repository. I copied it onto my EFI partition, so it could be accessed at boot. It went a little something like this:

    sudo mkdir -p /Volumes/ESP
    sudo mount -t msdos /dev/disk0s1 /Volumes/ESP
    sudo cp ~/Downloads/Shell_Full.efi /Volumes/ESP/EFI/tools/ShellX64.efiCode language: JavaScript (javascript)

    This meant that at boot, I’d have an EFI shell available to execute commands manually.

    Rather than having to manually select the EFI shell every time, I installed rEFInd to make life easier. This boot manager would allow me to configure an automatic script to disable the bad RAM before macOS even loads.


    After installing rEFInd and mounting the EFI partition again, I placed the RAM-disabling utility (disable-ram-area.efi) into the correct folder by running sudo cp disable-ram-area.efi /Volumes/ESP/EFI/refind/


    Now came the fun part—writing a script (startup.nsh) that would automatically execute disable-ram-area.efi with the right parameters every time the system booted.

    My bad RAM addresses (from MemTest86) were 0x1608F80780x1628FFCB4 (~32MB). To ensure alignment, I slightly expanded the range to 0x1608F70000x162A00000. This ensures that the entire faulty region is covered and avoids instability caused by page misalignment.

    My final startup.nsh script looked like this:

    echo Running disable-ram-area.efi
    
    # Locate the EFI partition (fs0:)
    set DISK fs0
    
    # Check if disable-ram-area.efi exists
    if not exist %DISK%\EFI\refind\disable-ram-area.efi then
        echo ERROR: Could not find disable-ram-area.efi in /EFI/refind/. Exiting...
        exit
    endif
    
    echo Found disable-ram-area.efi on %DISK%
    stall 1000000
    
    # Run the memory fix
    %DISK%\EFI\refind\disable-ram-area.efi 0x1608F7000 0x162A00000
    
    echo Starting macOS
    stall 1000000
    
    # Locate macOS bootloader (fs3:)
    set MACOS fs3
    
    # Check if boot.efi exists
    if not exist %MACOS%\System\Library\CoreServices\boot.efi then
        echo ERROR: Could not find boot.efi in fs3:. Exiting...
        exit
    endif
    
    echo Found boot.efi on %MACOS%
    stall 1000000
    
    # Boot macOS
    %MACOS%\System\Library\CoreServices\boot.efiCode language: PHP (php)

    This script does the following:

    1. Finds the EFI partition (fs0:) where disable-ram-area.efi is stored.
    2. Runs disable-ram-area.efi with my calculated bad RAM range.
    3. Finds the correct fsX: partition that contains boot.efi (macOS bootloader).
    4. Boots macOS, now with the bad RAM disabled.

    After saving startup.nsh to /Volumes/ESP/EFI/refind/, I rebooted into rEFInd, selected “EFI Shell”, and ran:

    fs0:
    cd EFI/refind
    startup.nsh

    It worked! The script executed, disabled my bad RAM, and booted into macOS.

    To make sure this runs automatically every time I turn on my Mac, I edited refind.conf and added:

    menuentry "Boot macOS with Defective RAM Disabled" {
        loader /EFI/tools/ShellX64.efi
        options "fs0:\EFI\refind\startup.nsh"
    }Code language: JavaScript (javascript)


    And to set it as the default boot option, I updated:

    default_selection "Boot macOS with Defective RAM Disabled"Code language: JavaScript (javascript)


    Now, every time I boot up my Mac, it automatically disables the bad RAM before macOS loads! Huge win!

    Now, this is just my story, please don’t take this as a tutorial. I rebuilt the entire process I have done through some scribbled notes, open tabs, and browser histories. I may have missed things or gotten them wrong and I’m too lazy to remount my EFI partition to verify that the scripts I shared here are the final ones I actually used.

    But if you find yourself in a similar position, hopefully this helps you with your own process of trying to save your MacBook.

  • Setting up Pibooth in 2024

    Setting up Pibooth in 2024

    I have an upcoming need for a photo booth next year 😏 so I started looking into some options to DIY rather than rent one or buy a pre-made version.

    The option I’m trying out right now is Pibooth, “A photobooth application out-of-the-box in pure Python.”

    https://pibooth.org/

    I’m going to set this up with an extra Raspberry Pi 400 I have that’s not doing anything, and see what we can do. Feel free to follow along!

    Since Pibooth only works on Raspbian Buster right now, I had to download an older version at https://downloads.raspberrypi.org/raspios_arm64/images/raspios_arm64-2021-05-28/

    I set up a few defaults I like to have for my Pis:

    # Updates, new software, and cleanup
    sudo apt update && sudo apt upgrade
    sudo apt install mc screen ack zsh locate git htop cockpit -y
    sudo apt autoremove
    
    # Add dotfile customizations. Sorry, it's currently private :D
    git clone git@github.com:emrikol/dotfiles.git
    cp -r ~/dotfiles/. ~/
    sudo usermod --shell /bin/zsh derrick
    zsh
    
    # Set up root access so I can SCP in from Transmit if I need to.
    sudo sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config
    sudo passwd root
    
    # Customize Raspberry Pi settings.
    sudo raspi-configCode language: Bash (bash)

    After installing cockpit I believe, I had an issue where the MAC address of my Wifi kept changing randomly on each reboot. I had to follow these instructions and add this to my /etc/NetworkManager/NetworkManager.conf:

    [device]
    wifi.scan-rand-mac-address=no

    Disable Swap to save on SD Card wear:

    sudo dphys-swapfile swapoff
    sudo dphys-swapfile uninstall
    sudo update-rc.d dphys-swapfile remove
    sudo apt purge dphys-swapfile -y
    sudo sysctl -w vm.swappiness=0Code language: Bash (bash)

    Install Log2RAM for the same reason:

    echo "deb [signed-by=/usr/share/keyrings/azlux-archive-keyring.gpg] http://packages.azlux.fr/debian/ bookworm main" | sudo tee /etc/apt/sources.list.d/azlux.list
    sudo wget -O /usr/share/keyrings/azlux-archive-keyring.gpg  https://azlux.fr/repo.gpg
    sudo apt update
    sudo apt install log2ram
    sudo sed -i 's/SIZE=40M/SIZE=64M/' /etc/log2ram.conf
    sudo sed -i 's/#SystemMaxUse=/SystemMaxUse=32M/' /etc/systemd/journald.conf
    sudo systemctl restart systemd-journald
    Code language: Bash (bash)

    From here, we should have my default “base” Raspberry Pi setup. And now, we can work on figuring out how to install Pibooth. According to the install docs, we need to run a few commands:

    $ sudo apt install "libsdl2-*"
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    Note, selecting 'libsdl2-mixer-dev' for glob 'libsdl2-*'
    Note, selecting 'libsdl2-image-dev' for glob 'libsdl2-*'
    Note, selecting 'libsdl2-gfx-dev' for glob 'libsdl2-*'
    Note, selecting 'libsdl2-gfx-doc' for glob 'libsdl2-*'
    Note, selecting 'libsdl2-mixer-2.0-0' for glob 'libsdl2-*'
    Note, selecting 'libsdl2-dbg:armhf' for glob 'libsdl2-*'
    Note, selecting 'libsdl2-dev' for glob 'libsdl2-*'
    Note, selecting 'libsdl2-doc' for glob 'libsdl2-*'
    Note, selecting 'libsdl2-ttf-dev' for glob 'libsdl2-*'
    Note, selecting 'libsdl2-net-2.0-0' for glob 'libsdl2-*'
    Note, selecting 'libsdl2-net-dev' for glob 'libsdl2-*'
    Note, selecting 'libsdl2-image-2.0-0' for glob 'libsdl2-*'
    Note, selecting 'libsdl2-2.0-0-dbgsym:armhf' for glob 'libsdl2-*'
    Note, selecting 'libsdl2-2.0-0' for glob 'libsdl2-*'
    Note, selecting 'libsdl2-gfx-1.0-0' for glob 'libsdl2-*'
    Note, selecting 'libsdl2-ttf-2.0-0' for glob 'libsdl2-*'
    libsdl2-2.0-0 is already the newest version (2.0.9+dfsg1-1+deb10u1).
    libsdl2-2.0-0 set to manually installed.
    Some packages could not be installed. This may mean that you have
    requested an impossible situation or if you are using the unstable
    distribution that some required packages have not yet been created
    or been moved out of Incoming.
    The following information may help to resolve the situation:
    
    The following packages have unmet dependencies:
     libsdl2-2.0-0-dbgsym:armhf : Depends: libsdl2-2.0-0:armhf (= 2.0.9+dfsg1-1+rpt1) but it is not going to be installed
    E: Unable to correct problems, you have held broken packages.Code language: JavaScript (javascript)

    Meh. Okay. Let’s just install all of them but the trouble package. Hopefully that won’t come back to bite us.

    sudo apt install libsdl2-mixer-dev libsdl2-image-dev libsdl2-gfx-dev libsdl2-gfx-doc libsdl2-mixer-2.0-0 libsdl2-dev libsdl2-doc libsdl2-ttf-dev libsdl2-net-2.0-0 libsdl2-net-dev libsdl2-image-2.0-0 libsdl2-2.0-0 libsdl2-gfx-1.0-0 libsdl2-ttf-2.0-0 -yCode language: Bash (bash)

    We did not install libsdl2-dbg and libsdl2-2.0-0-dbgsym

    I’m thinking about adding printer support, so I’ll go ahead and install CUPS: sudo apt-get install cups libcups2-dev

    And we might as well install OpenCV sudo apt-get install python3-opencv

    Installing gphoto2:

    cd ~
    git clone https://github.com/gonzalo/gphoto2-updater
    cd gphoto2-updater
    sudo ./gphoto2-updater.shCode language: Bash (bash)

    Now for pibooth: pip3 install "pibooth[dslr,printer]"

    Aww yeah! Success!

    Now all I need to do is customize it. Maybe we’ll have another post at a later time.