Raspberry Pi 4 Slots
Four-slot Raspberry Pi cluster board starts at $80 Mar 4, 2020 — by Eric Brown — 65466 views IPTerra’s “CloverPi” cluster board starts at $80 on Kickstarter, supporting up to 4x Raspberry Pi SBCs with power, switches, LEDs, and network headers for each plus a 5-port GbE switch with uplink port.
Recently, Tomasz Mloduchowski posted a popular article on his blog detailing the steps he undertook to get access to the hidden PCIe interface of Raspberry Pi 4: the first Raspberry Pi to include PCIe in its design. After seeing his post, and realizing I was meaning to go buy a Raspberry Pi 4, it just seemed natural to try and replicate his results in the hope of taking it a bit further. I am known for Raspberry Pi Butchery, after all.
Before I tried desoldering anything, I set up my Pi for remote use; enabling SSH, WiFi, serial UART+ boot messages. The USB ports on the Pi board will not function after this modification, so this is super important.
As Tomasz lays out in his article, we need to remove the VL805 USB3 chip in order to access the PCIe interface. I used a hot air soldering station at low volume and medium-high temperature, with small nozzle head in order to not disturb the components nearby. I used flux along the edges and after a while the chip came away.
- Eau 4 Slot Raspberry Pi 4 Cluster Case, 4 Layers Acrylic Case Pi Rack Case Stackable Case for Raspberry Pi 3B+, Raspberry Pi 3/2 Model B, Raspberry Pi 3 Clear Case with Cooling Fan Eau - We offer customers a replacement warranty and money back guarantee!
- Raspberry Pi 4 Storage, RAM, and Pricing. Q: Since the 3rd generation Raspberry Pi could only support a max of a 32GB micro SD card, will the Raspberry Pi 4 have any limitations on the maximum.
- Eau 4 Slot Raspberry Pi 4 Cluster Case, Raspberry Pi 3 Clear Case with Cooling Fan, 4 Layers Acrylic Case Pi Rack Case Stackable Case for Raspberry Pi 3B+, Raspberry Pi 3/2 Model B by EAU 4.0 out of 5 stars 11 ratings.
I tried to remove the solder from the large pad by mixing some low-temperature solder paste into whats there, but it’s not needed. Just cover it with capton or poor electrical tape. I used electrical tape just to make seeing the small wires I’d be soldering above it easier to see.
The VL805 datasheet is confidential which makes posting parts of it here tricky. However, an image search for “VL805-Q6 QFN68” may yield interesting results for those interested in finding out more. The pins we are interested in, are as follows (note differing polarity from Tomasz’s work):
I used 0.1mm enameled wire, with each differential pair cut to nearly the same length. Tinning the end of the wire by scraping with a knife and dipping in a molten solder ball makes soldering to the pads we need easier. Holding the wires down with kapton tape, and using flux, with the smallest iron tip I had made the job just bearable under a microscope.
Raspberry Pi 4 Ram Slot
The rather untidy looking result:
I was using a cheap PCIe riser as my “first interface” which was then to be connected to a PCIe switch card, which would then enable another 4 PCie slots. This first socket, as Tomasz mentioned in his article, needs the PCIe Reset pin pulled to 3v3, and both Reset and Wake signal traces to the USB socket cut. Note that whilst we now know where the PCIE Reset line is on the Pi, I have not needed to connect this as yet.
The first attempt to boot with this setup resulted in the Pi not managing to boot at all. After some wiggling of the PCIe slot, the raspberry Pi booted, but no devices were shown when running lspci (lspci can be installed via apt-get). The third attempt, however, after some professionalwiggling of the PCIe slot, resulted in success! A booted Pi, with a PCIe switch!
However, no devices were detected beyond the ASM1184e switch. Even a USB3 PCIe card using the same VL805 chip that I removed refused to detect. Running dmesg on the pi to get some driver details, I saw that whilst the PCIe link was active, and some busses were being assigned to the switch – it said that devices behind the bridge would not be usable due to bus IDs.
The great thing about linux is that the source code is just there to dig into, and after finding where that “devices behind bridge are unusable” warning is printed I further discovered that the range of assignable busses can be limited by the Device Tree linux uses.
Device trees are simply a description of the hardware which is passed to the Linux kernel on boot. It has all the devices listed; their driver compatabilities, memory mappings, and configuration. It is particularly useful for describing peripherals which may not be discoverable via conventional means. On the root directory of your Raspbian Raspberry Pi boot SD volume, you will find bcm2711-rpi-4-b.dtb – which is the Compiled Device Tree binary. This binary blob is not user readable, but thankfully we can use the Device Tree Compiler to decompile it into a readable form. I did all of this within Windows Subsystem for Linux.
That command will decompile into bcm2711-rpi-4-b_edit.dts. Searching this file for “pci” we find an entry for the PCIe – and it has a definition for bus-range which limits the bus IDs from 0 to 1. I change the entry from “<0x0 0x1>” to “<0x0 0xff>” and recompile to the binary form, and place that on the Raspbian SD card overwriting the default.
Command to recompile:
Raspberry Pi 4 Slots Free Play
4 More PCIe busses!
And then I plugged some VL805-based USB3 cards in (one with a chained Network Interface). The setup looks as follows:
Raspberry Pi 4 Slot Machine
I have my Motorola LapDock USB hub connected to the Pi via the PCIe USB controller. The keyboard and trackpad work great!
I have a SATA controller based on a JMicron JMB363, which is detected correctly but there is no driver to load. This will require some linux driver/kernel fiddling to get a driver loaded correctly – but it’s very promising!
I also have tried some other fairly hilarious setups, including the following with a Radeon HD 7990 GPU, and another with a GTX 1060.
I’ll leave this here for now, whilst I read up on the Linux driver stack and how to build kernels for the raspberry pi 🙂
Thanks for reading! You can find me, as always, over on twitter @domipheus. Additionally, thanks to Tomasz Mloduchowski for his previous blogs which spurred my interest! I don’t fully understand why Tomasz had kernel panics using a VL805 board, but maybe it’s something to do with the Device Tree, and the fact I also had a PCIe switch.
Updates: