Jump to content

Gigabyte Titan Ridge on Proxmox/OSX baremetal (WIP)


Recommended Posts

  • Supervisor
9 minutes ago, iGPU said:

 

Once you flash a TB card, it won't work in Windows.

 

Good thank you for this info.

i have also with v50 original firmware and i cant see disk connected also

i see all devices on system properties as it should be..i think

i would like to ask:

1) with a not proper tb header jumper cable, what happens?

2) with a not proper usb 2 cable connected on internal usb header and on internal motherboard header, what happens?

i have no experiences at all with thunderbolt so any informations could be very helpfull

thank you

  • Like 1
Link to post
Share on other sites
  • Supervisor

@iGPU i have forgotten to say i am on latest 5.77.1 zen2 kernel on proxmox

i have said this before but i would like to say again( i do not know if it has a differnt behaviour from previous one

  • Like 1
Link to post
Share on other sites
10 hours ago, fabiosun said:

 

Good thank you for this info.

i have also with v50 original firmware and i cant see disk connected also

i see all devices on system properties as it should be..i think

i would like to ask:

1) with a not proper tb header jumper cable, what happens?

2) with a not proper usb 2 cable connected on internal usb header and on internal motherboard header, what happens?

i have no experiences at all with thunderbolt so any informations could be very helpfull

thank you

 

If the mobo does not have a TB header, you must jumper 2 pins on the TB AIC for proper power. (I can supply photos or links on how to do this, if you'd like.) But since you have good appearance on IORegistryExplorer, I would not change whatever you've done to your AIC.

 

Mobos that have header also have BIOS settings for TB, so this is ideal. However, even without headers and BIOS support, TB function is seen with flashed firmware.

 

As for the USB2 connector on TB AIC, this is not necessary for TB function. When connected, it is to provide more power for charging devices connected to TB ports. So this connection can be ignored.

 

*** 

 

I must make a correction for above post showing the attachment on my MSI mobo of TB at SF0. It is not; it was late and I was not carefully studying IORegistryExplorer. On review this morning, I found the NHI section (15d2 on AR) is attaching at 10,7 and the USB section (15d4 on AR) is attaching at S88. So these are not grouped together and cannot be treated with an SSDT (and certainly explains why the SSDTs I tried for SF0 didn't work). I will move the AR AIC from slot 4 to slot 2 and see if this will group the TB device differently (the water cooled GPUs only take up 1 slot, so room next to slot 1). Moving to slot 2 didn't help; it gave the exact same results with NHI and USB being split over 2 locations as shown below.

 

 15d2.png.3559442189ba6fe5f0a7af29b0d931a3.png

 

15d4.png.514b89591dc45442d466e96710a84e6a.png

 

 

Edited by iGPU
added results for slot 2.
Link to post
Share on other sites
9 hours ago, fabiosun said:

@iGPU i have forgotten to say i am on latest 5.77.1 zen2 kernel on proxmox

i have said this before but i would like to say again( i do not know if it has a differnt behaviour from previous one

 

I'm not on zen2. I don't know how to change to zen2 settings. (If this zen2 is from a patch; I don't know how to patch things.)

 

I ran "apt update" and  "apt dist-upgrade", and that led to the latest update shown in Spoiler below.

 

Command: "pveversion -v"

Spoiler

proxmox-ve: 6.2-1 (running kernel: 5.4.44-2-pve)
pve-manager: 6.2-10 (running version: 6.2-10/a20769ed)
pve-kernel-5.4: 6.2-4
pve-kernel-helper: 6.2-4
pve-kernel-5.4.44-2-pve: 5.4.44-2
pve-kernel-5.4.34-1-pve: 5.4.34-2
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.4-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.16-pve1
libproxmox-acme-perl: 1.0.4
libpve-access-control: 6.1-2
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.1-5
libpve-guest-common-perl: 3.1-1
libpve-http-server-perl: 3.0-6
libpve-storage-perl: 6.2-5
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.2-1
lxcfs: 4.0.3-pve3
novnc-pve: 1.1.0-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.2-9
pve-cluster: 6.1-8
pve-container: 3.1-12
pve-docs: 6.2-5
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-2
pve-firmware: 3.1-1
pve-ha-manager: 3.0-9
pve-i18n: 2.1-3
pve-qemu-kvm: 5.0.0-11
pve-xtermjs: 4.3.0-1
qemu-server: 6.2-11
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-1
zfsutils-linux: 0.8.4-pve1

 

Link to post
Share on other sites
  • Supervisor
1 hour ago, iGPU said:

 

If the mobo does not have a TB header, you must jumper 2 pins on the TB AIC for proper power. (I can supply photos or links on how to do this, if you'd like.) But since you have good appearance on IORegistryExplorer, I would not change whatever you've done to your AIC.

 

Mobos that have header also have BIOS settings for TB, so this is ideal. However, even without headers and BIOS support, TB function is seen with flashed firmware.

 

As for the USB2 connector on TB AIC, this is not necessary for TB function. When connected, it is to provide more power for charging devices connected to TB ports. So this connection can be ignored.

 

*** 

 

 

 

 

 

 

sorry, maybe I explained myself wrong. I just wanted to say, in case I had hurt the jumper between the two pins or if the internal USB cable didn't work ... what happens?

 

I ask this because I can't see any peripheral in windows if I connect it to the thunderbolt port with a type C cable (USB stick or disk)

 

14 minutes ago, iGPU said:

 

I'm not on zen2. I don't know how to change to zen2 settings. (If this zen2 is from a patch; I don't know how to patch things.)

 

I ran "apt update" and  "apt dist-upgrade", and that led to the latest update shown in Spoiler below.

 

Command: "pveversion -v"

  Reveal hidden contents

proxmox-ve: 6.2-1 (running kernel: 5.4.44-2-pve)
pve-manager: 6.2-10 (running version: 6.2-10/a20769ed)
pve-kernel-5.4: 6.2-4
pve-kernel-helper: 6.2-4
pve-kernel-5.4.44-2-pve: 5.4.44-2
pve-kernel-5.4.34-1-pve: 5.4.34-2
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.4-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.16-pve1
libproxmox-acme-perl: 1.0.4
libpve-access-control: 6.1-2
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.1-5
libpve-guest-common-perl: 3.1-1
libpve-http-server-perl: 3.0-6
libpve-storage-perl: 6.2-5
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.2-1
lxcfs: 4.0.3-pve3
novnc-pve: 1.1.0-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.2-9
pve-cluster: 6.1-8
pve-container: 3.1-12
pve-docs: 6.2-5
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-2
pve-firmware: 3.1-1
pve-ha-manager: 3.0-9
pve-i18n: 2.1-3
pve-qemu-kvm: 5.0.0-11
pve-xtermjs: 4.3.0-1
qemu-server: 6.2-11
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-1
zfsutils-linux: 0.8.4-pve1

 

🙂 🙂

 

you have given a like also to this post 🙂

 

  • Like 1
Link to post
Share on other sites
  • Supervisor

IMG_0945.jpg.80a7a40d533b3dc8ddfff20a971a8fb8.jpg720703B0-E29F-4B3C-993F-48571B3AA710.JPG.9bdda6f0997617959f408db096334367.JPG0DDED72E-DD48-4B21-8B31-36BCA73C3E31.JPG.0f53978d91c9702b61faabaf2ab2df0e.JPGIMG_0954.jpg.e4a8a83c4c8d99ae2c74223cea76c49a.jpg

 

and this one are some steps to patch Titan Ridge card
I have followed @iGPU links to do in a safe way
By the way.. I have (maybe ) discovered a trick with that eprom programmer about 3.3, 5 v situation 

I have not desoldered any stuff because it is over my skill

  • Like 2
  • +1 1
Link to post
Share on other sites
  • Supervisor

@iGPU

Spoiler

2020-08-01 19:42:29.574330+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform) SSDT 0x000000007D23F000 000899 (v02 CASEY  TBTTITAN 00000000 INTL 20200110)

2020-08-01 19:42:29.574331+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform) SSDT 0x000000007D23F000 000899 (v02 CASEY  TBTTITAN 00000000 INTL 20200110)

2020-08-01 19:42:29.574657+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform)

2020-08-01 19:42:29.574658+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform)

2020-08-01 19:42:29.584499+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform) ACPI Error:

2020-08-01 19:42:29.584499+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform) ACPI Error:

2020-08-01 19:42:29.584553+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform) [PXSX]

2020-08-01 19:42:29.584553+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform) [PXSX]

2020-08-01 19:42:29.584580+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform)  Namespace lookup failure, AE_NOT_FOUND

2020-08-01 19:42:29.584581+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform)  Namespace lookup failure, AE_NOT_FOUND

2020-08-01 19:42:29.584752+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform)  (20160930/dswload-292)

2020-08-01 19:42:29.584752+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform)  (20160930/dswload-292)

2020-08-01 19:42:29.584860+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform) ACPI Exception: AE_NOT_FOUND,

2020-08-01 19:42:29.584861+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform) ACPI Exception: AE_NOT_FOUND,

2020-08-01 19:42:29.584993+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform) During name lookup/catalog

2020-08-01 19:42:29.584993+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform) During name lookup/catalog

2020-08-01 19:42:29.585107+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform)  (20160930/psobject-310)

2020-08-01 19:42:29.585107+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform)  (20160930/psobject-310)

2020-08-01 19:42:29.585223+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform) ACPI Exception: AE_NOT_FOUND,

2020-08-01 19:42:29.585224+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform) ACPI Exception: AE_NOT_FOUND,

2020-08-01 19:42:29.585357+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform) (SSDT:TBTTITAN) while loading table

2020-08-01 19:42:29.585357+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform) (SSDT:TBTTITAN) while loading table

2020-08-01 19:42:29.585514+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform)  (20160930/tbxfload-319)

2020-08-01 19:42:29.585515+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform)  (20160930/tbxfload-319)

2020-08-01 19:42:29.585626+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform) ACPI Error:

2020-08-01 19:42:29.585626+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform) ACPI Error:

2020-08-01 19:42:29.585679+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform) 1 table load failures, 4 successful

2020-08-01 19:42:29.585680+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform) 1 table load failures, 4 successful

2020-08-01 19:42:29.585834+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform)  (20160930/tbxfload-342)

2020-08-01 19:42:29.585834+0200 0x71       Default     0x0                  0      0    kernel: (AppleACPIPlatform)  (20160930/tbxfload-342)

 

  • Like 2
Link to post
Share on other sites

@fabiosun, @iGPU - I see you guys are having some TB3 fun. I have this card on my motherboard (Designare TRX40) installed in slot 2. I never tried to  pass it through to one of VMs, as I don't have a device yet that uses it (besides my TB3 monitor in Linux and charging, and that works just fine).  I am hoping that if I get a PCIE external GPU, then I can pass that through.

 

Flashing the eprom and trying to pass the entire TB3 seems like a very complicated exercise. Hope it works. Let me know if I can help, but I don't have eprom flashing device at the moment.

 

 

Link to post
Share on other sites
  • Supervisor
3 hours ago, meina222 said:

 

Flashing the eprom and trying to pass the entire TB3 seems like a very complicated exercise. Hope it works. Let me know if I can help, but I don't have eprom flashing device at the moment.

 

 

 

https://www.amazon.it/gp/product/B07SNTL5V6/

with above was very simple to do..I had zero experience in this task

  • Like 1
Link to post
Share on other sites

@fabiosun, thanks for creating this thread. I am interested in this topic too. Before I study this, which may take me a while, I have a couple of questions:

 

1. If you flash the Titan Ridge card with a macOS compatible firmware, does it render it useless in Linux? I don't have Windows, so I'm less concerned about it, but I certainly don't want to lose host support.

2. As a simpler goal to getting macOS VM to use TB3 devices - as TB3 devices are supposed to act like a PCIE devices - has anyone successfully passed through a TB3 device connected to the card without passing the TB3 controller itself? I don't have an external GPU or TB3 disk to try yet, but as all my PCIE slots are taken, future expansions would have to inevitably happen via TB3 or USB.

 

Thanks.

Link to post
Share on other sites
  • Supervisor

@meina222

it is possible to backup your original firmware and in the worst case flash back again

however with flashed firmware i can see disk usb type c connected on tb port under linux

i have no tested thunderbolt device because i have not

the journey is started..and i hope we have success to see device in osx👍🏻

  • Like 1
Link to post
Share on other sites

@fabiosun

I got ahold of an unflashed ASUS TR AIC. I placed this native card in slot 2 and added SSDT for a re-boot test. This card, like the Alpine Ridge, splits into the same two sections for NHI and USB. The NHI device (0x15eb) is located at pci,bridge10,7; the TB USB device (0x15ec) is located at S88@11 (not shown). It's not clear why on this MSI Creator mobo, it shows up split, while on your MSI mobo, fabiosun, it does not. Curious.

 

(I will flash the card later this week and re-test.)

 

The location of the NHI device was at pci,bridge10,7; an SSDT injected here using this code snippet:

Spoiler

 


    Scope (_SB.PCI0)
    {
        Device (UPSB)
        {
            Name (_ADR, 0x00100007)  // _ADR: Address
   
            Device (DSB0)
            {
                Name (_ADR, Zero)  // _ADR: Address
            }

            xxxxxxx
         }
    }

 

 

becomes this entry in IORegistryExplorer (but is non-functional):

511018231_UPSB@107.png.3e1a85bf94ae0744d579d6eadb5d1c15.png

 

 

And, when looking at the PCI section of SystemInfo below, the SSDT file injected "Slot-2" for the NHI section. The remainder of the SSDT injected descriptions, such as "Titan Ridge Thunderbolt Controller", do not appear because there is no proper TB tree. Not surprisingly, no driver is working.

 

Meanwhile, both the NHI and USB sections, in the PCI pane below, have entries that say "JHL6540". This is a purposeful error to 'tag' NHI and USB from another direction. Devices can be labelled from either SSDT files or from the DeviceProperties section inside the config.plist file. Here, labels from Alpine Ridge were used, so as to know that the label was from DeviceProperties and not the SSDT. 

 

PCI-slots.png.1208e400478b99e650ad8e0ad4c37d72.png

 

The USB device, while it does have a driver, nevertheless shows up under the SystemInfo/USB pane. Unfortunately, there is no USB function.

PCI-USB.png.05dfcee63a03645bcc27cde2b291167b.png

 

***

 

@meina222

I originally had a GB TRX40 Designare but ended up returning it for faults (which in retrospect was probably due to a bad CPU). Anyhow, my earlier TB experiments were unsuccessful on that mobo too, despite it being designed for TB with a proper header and BIOS entries for TB.

 

 

Edited by iGPU
Link to post
Share on other sites
  • Supervisor

in this weekend I have flashed many firmware found on the net (GitHub)

I will put here the exact name of these files but not the direct link that you can find easily googling

 

1)  TitanRidgeMacOSFirmware.bin (DSM2 firmware I think)

2)  TitanRidgeNVM23-Elias64Fr-Mod.bin (Eliasfr caseSj firmware I think)

3)  TitanRidgeNVM43-Elias64Fr-Mod.bin  (Eliasfr caseSj firmware I think)

4)  original v43 firmware (stock in my board)

5)  original v50 firmware (updated via Gigabyte windows updater utility)

 

fast analysis :

with all firmware I have the same driver loaded in windows, but with all of these I can't see any device connected to TB output, I have no thunderbolt devices and I have tried only with two different USB type c devices

 

with 1) in OSX I see only 15eb if I do not boot before in windows and I have a pretty full ioreg with all we need..with bad naming

 

with 2) I have the best result in OSX, no need to boot in windows before and I have the pretty same IOREG (full of stuff) with always bad naming 

 

with 3) I have a minimal ioreg similar to original not patched version (identical, and this le me think it is an original firmware)

 

with 4) and 5) the same minimal ioreg in OSX

 

I have verified thanks to @thenightflyer for this that I have the same his ioreg in many parts except for naming and some voices, I think because he is on x299 system which support better ssdt part

Any idea to how solve this puzzle will be accepted 🙂

 

attached 2 images for clarification, on the right x299 working ones

foto2.jpg

foto1.jpg

  • Like 3
Link to post
Share on other sites

I was about to buy a Gigabyte Titan Ridge with flashed firmware. But i have just read in this thread that if you use a Titan Ridge with flashed firmware this does not work properly under Windows. Did I get it right? If yes, i will not proceed with the purchase and will remain with the original card.

 

Thank you!

Link to post
Share on other sites

Thunderbolt 3 and ASRock TB3 cards 

 

I queried ASRock Support regarding their TB3 card and working under TRX40 Creator motherboards. Here's the response.

 

Hello,

 

Our TRX40 models have no header for a Thunderbolt add-in card. No way to add Thunderbolt support to these boards, sorry.

 

Kind regards,

ASRock Support

 

ASRock Europe B.V.

Bijsterhuizen 1111

6546AR Nijmegen

The Netherlands

Edited by Driftwood
  • Like 1
Link to post
Share on other sites
2 hours ago, Driftwood said:

Thunderbolt 3 and ASRock TB3 cards 

 

I queried ASRock Support regarding their TB3 card and working under TRX40 Creator motherboards. Here's the response.

 

Hello,

 

Our TRX40 models have no header for a Thunderbolt add-in card. No way to add Thunderbolt support to these boards, sorry.

 

 

 

If jumper on AIC is placed, then TB AIC can usually be used on mobo without TB header, but you need to flash firmware for macOS. These steps are never condoned, nor discussed, by manufacturers.

Link to post
Share on other sites
6 hours ago, fabiosun said:

in this weekend I have flashed many firmware found on the net (GitHub)

I will put here the exact name of these files but not the direct link that you can find easily googling

 

1)  TitanRidgeMacOSFirmware.bin (DSM2 firmware I think)

2)  TitanRidgeNVM23-Elias64Fr-Mod.bin (Eliasfr caseSj firmware I think)

3)  TitanRidgeNVM43-Elias64Fr-Mod.bin  (Eliasfr caseSj firmware I think)

4)  original v43 firmware (stock in my board)

5)  original v50 firmware (updated via Gigabyte windows updater utility)

 

fast analysis :

with all firmware I have the same driver loaded in windows, but with all of these I can't see any device connected to TB output, I have no thunderbolt devices and I have tried only with two different USB type c devices

 

with 1) in OSX I see only 15eb if I do not boot before in windows and I have a pretty full ioreg with all we need..with bad naming

 

with 2) I have the best result in OSX, no need to boot in windows before and I have the pretty same IOREG (full of stuff) with always bad naming 

 

with 3) I have a minimal ioreg similar to original not patched version (identical, and this le me think it is an original firmware)

 

with 4) and 5) the same minimal ioreg in OSX

 

I have verified thanks to @thenightflyer for this that I have the same his ioreg in many parts except for naming and some voices, I think because he is on x299 system which support better ssdt part

Any idea to how solve this puzzle will be accepted 🙂

 

attached 2 images for clarification, on the right x299 working ones

foto2.jpg

foto1.jpg

 

I too use on X299. Writing SSDT is not the hard part.

 

fabiosun, I'd like to see more complete IORegistryExporer TB tree of your best firmware flash (I think your best is #2, TitanRidgeNVM23-Elias64Fr-Mod.bin ?). In my previous flashing, this has been the best firmware to date for TR TB3 cards.

 

Do not enter a search criteria in order to see whole tree, but do click on triangles to collapse the tree at major nodes. This will allow the screen shot can show details of whole tree. The other option is to simply upload the IORegistryExplorer file so I can study. Thanks.

Link to post
Share on other sites
  • Supervisor
29 minutes ago, iGPU said:

 

I too use on X299. Writing SSDT is not the hard part.

 

fabiosun, I'd like to see more complete IORegistryExporer TB tree of your best firmware flash (I think your best is #2, TitanRidgeNVM23-Elias64Fr-Mod.bin ?). In my previous flashing, this has been the best firmware to date for TR TB3 cards.

 

Do not enter a search criteria in order to see whole tree, but do click on triangles to collapse the tree at major nodes. This will allow the screen shot can show details of whole tree. The other option is to simply upload the IORegistryExplorer file so I can study. Thanks.

I will do more 🙂

attached ioreg with patched 23 firmware and 50 untouched 

tb_ioreg.zip

  • Like 3
Link to post
Share on other sites

fabiosun, thanks for sharing.

 

Here are your two trees. I've condensed and re-structured the trees for better clarity. Note how the TB items (NHI at 0x15eb and USB at 0x15ec) fall under the SF0 device, but in parallel with other devices, such as USB Audio and SATA drives. This is not good: SFO is not a good attachment site for TB (but we seem to have no control over this). And, as I've mentioned previously, I've never gotten an SSDT to re-name or inject properties into any device contained within SF0.

 

fabiosun's v23 (left) and v50 (right):

FW50.png.b23bc1854f3fe0ad699785054ab9d185.png                         FW23.png.dc4d6445eb268781024eff6afadb9e86.png           

 

 

Below is a detail from v23 to better describe what we're seeing. While the NHI and USB devices are present, the associated pci-bridges are not. Obviously missing is the essential UPSB, the pci-bridge that contains all of the TB nodes.

 

This is the same problem I described months ago. Until we can get the entire PCI device passed-through, we cannot get functionality from that PCI device. This holds true for USB, BT/Wifi, FireWire and TB PCI devices. This is a Linux problem. So while you're seeing portions of the TB tree, it is not cohesive and most likely will not function (in this latter aspect, I hope I am wrong).

 

fabiosun's v23 (detail):

v23-detail.png.0b373e86899232d1ba02ee1869335579.png

 

 

***

 

Shown for more clarity (below) is the TB tree I see on my GB Z390 Aorus Xtreme Waterforce mobo, where I flashed the chip on the mobo. To review, DSB0 is the main TB device, NHI and its associated ports. DSB1 and DSB4 are for attached external devices connected to the TB AIC, such as external TB drives, TB hubs or TB audio devices. DSB2 is for USB devices; here you'll see USB-C attached devices.

 

In the image below, DSB4 is expanded (as compared to DSB1, which is not expanded) since there is a TB drive ("OWC Aura P12 2TB") connected to it. This drive has several partitions, two of which are mounted ("MBA-500" and "ExtStorage").

 

Note how all TB branches fall under UPSB (DSB0-DSB4). The UPSB node (bridge) is missing on the above 'mini-trees'. UPSB in turn falls under the main device, here labelled RP01 (RP01 is equivalent to SF0). UPSB is a critical pci-bridge that is not being passed. DSB1 and DSB4 are also not present in the above 'mini-trees', and so no external TB devices or hub can connect to the TB AIC.

 

Proper TB tree GB Z390 Aorus Xtreme Waterforce build:

ProperTBtree.png.37ab5e65b6daf7ff6e598c9256562fda.png

 

 

Edited by iGPU
  • Like 1
Link to post
Share on other sites
  • Supervisor

a little update:

V23 flashed firmware allows to see my Titan ridge on windows in all PCIE slot I put  it (tested on 4 and 2)

Linux does not see it in slot 2

Only in slot 4 I can see and pass two pcidevice (TB and USB)

with the help of @Gengik84I am trying a new way to proceed

DSDT way 🙂

 

  • Like 2
  • Cross Finger 1
Link to post
Share on other sites

I haven't had time to do much research on this topic, but here a few things that I came across:

 

1st check:

 

https://www.kernel.org/doc/Documentation/vfio.txt

 

In particular this section:

 

This device is behind a PCIe-to-PCI bridge [4]_, therefore we also
need to add device 0000:06:0d.1 to the group following the same
procedure as above.  Device 0000:00:1e.0 is a bridge that does
not currently have a host driver, therefore it's not required to
bind this device to the vfio-pci driver (vfio-pci does not currently
support PCI bridges).

 

When I did lspci and checked my TB group id, and then did  ls -l /sys/bus/pci/devices/[id]/iommu_group/devices

 

the structure was very similar to what's described there. Essentially the 'leaf' devices are behind a PCI bridge. I don't have the Proxmox console handy now but I will paste the result of this later tonight. I don't think I am saying anything new here - still worth trying and see what happens.

 

I think it would help to reach out to Alex Williamson who is the maintainer of VFIO in the linux kernel

 

https://www.linkedin.com/in/alexwilliamson

 

He might provide helpful tips, or simply tells us not to waste our time as it is not supported. Or this could prompt someone on  the Linux kernel team to look into emulating PCI bridge devices. At any rate, he seemed helpful on a reddit board where a user tried to achieve a simpler goal - passthrough a leaf PCIE video card through thunderbolt to his laptop. As you can see this already turned out to be non-trivial at the time

 

You can read the full thread here and see that he provided input:

 

 

  • Like 2
Link to post
Share on other sites
  • Supervisor

@iGPU

if we were able to rename in sf0 tree it should be useful to our goal?

thanks to gengik84 I have some renaming working on sf0 tree folder

 

 

I think also we have to study how to add additional bridges on qemu cfg files

 

maybe main problem is there for us

  • Like 1
  • +1 1
Link to post
Share on other sites
9 hours ago, fabiosun said:

@iGPU

if we were able to rename in sf0 tree it should be useful to our goal?

thanks to gengik84 I have some renaming working on sf0 tree folder

 

 

I think also we have to study how to add additional bridges on qemu cfg files

 

maybe main problem is there for us

 

I agree, problem order is:

 

1. passing bridges in VM

2. rename SF0

 

But problem #2 varies with mobo and setup.

 

Until I added a 3rd NVMe drive to the MSI mobo, TB was added to SF0 (as were the 2 NVMe drives). However, once I added a 3rd NMVe drive (filling all NVMe slots), all NVMe drives moved out of SF0 into their own devices inside the PCI0 tree. This has now left SF0 un-populated. Now, when I add the TB card, the two parts, while separated into different devices since there is no pci-bridge to bind them, are no longer inside SF0. Instead, these 2 parts are inside the PCI0 tree, leaving them accessible to a simple SSDT (once problem #1 is solved). 

 

So there seems to be curious behavior with the SF0 device, depending on number of occupied NVMe slots.

 

NVMe Drives (the 3rd drive contains Win10 and not visible; renaming RP0x done with an SSDT).

1196101751_NVMeDrives.png.c2c1aa9eb6cbbe03114ec6b29c3c4010.png

 

Un-populated SF0:

SF0-unpopulated.png.7bb96f9fef175a3a817ffec1075fba8e.png

 

  • Like 1
Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.