pci_enable_device() fails after remove/rescan

Question

I have here Linux 4.4 (I used to work on an older kernel which failed in the same way) with a PCIe connected FPGA device and a driver for it which are both of my own design. These have been working well under normal conditions, but now I try to make them work in hotplug conditions. This isn't actual hardware hotplug, what I have been trying is the usual echo 1 >remove in the device's sysfs directory and echo 1 >/sys/bus/pci/rescan afterwards.

After the device reappears, my driver's initialization calls pci_enable_device() which fails while logging:

otscan 0000:02:00.0: can't enable device: BAR 0 [mem 0xf7e01000-0xf7e013ff] not claimed
otscan 0000:02:00.0: can't enable device: BAR 1 [mem 0xf7e00000-0xf7e00fff] not claimed
otscan 0000:02:00.0: can't enable device: BAR 2 [mem 0xf0200000-0xf020ffff 64bit pref] not claimed

(Normally it would stop after the first unclaimed resource, but I have modified it to go on and confirm that in fact all BARs are unclaimed.)

"Not claimed" here means that the struct resource exists but has no parent, which from what I gather is caused by request_resource() never having been called on it. I do not think this is a driver issue as the initialization routine doesn't get to do lot before it aborts due to the failure to enable the device.

This leaves the FPGA (Altera Cyclone V with hard IP PCIe core) and something I might have done wrong there, such as mishandling bus reset somehow. Other PCIe device in that computer work when replugged through sysfs.

I have been digging around for a while and still haven't figured out what why my device is treated differently by Linux. What possible property of my device could have Linux decide to not call request_resource() on my device's BARs?

Mesmerism

I have seen the same problem myself in the past and thought it might have been related to the size of the resources (I had a 512KiB region). But yours are smaller. I never got to the bottom of it. – Bile

Answer 1

It looks like I found the cause. I have left the class code as 0 (which is invalid) in the PCIe core configuration which worked fine when the device was present at boot. Putting in a sensible value (0x40000 for multimedia video device in my case, 0xff0000 for "unregistered device" also worked) also made it work on hotplug.

It appears Linux only partially handles devices with a 0 class code.

Mesmerism

Exactly the same problem with cyclone V and an IMX6 linux system. Thanks for saving us quite some time:) – Glassy

Answer 2

the problem seems to be incorrectly defined class in the PCIe configuration space of the FPGA's PCIe Core. make sure upper byte of CLASS REGISTER is st to something different than 0

we had a similar problem when looking through dmesg we got: "can't enable device: BAR 0 ... ... not claimed" followed by "pci_enable_device failed"

Alfaro

Answer 3

It is indeed the class code that is not set correctly. I recommend setting 0x058000 which corresponds to a memory controller.

Plastid

Answer 4

to jump in on this, for people having similiar issues. This error can also be caused by having an incorrect device tree(.dtsi)

I was getting same issue. can't enable device: BAR 0 [mem 0x839000000000-0x8390000fffff 64bit] not claimed

My solution was to add the memory range to the linux device tree, dtsi. The memory range needs to be available in the device tree. If its not, you can get this same error. which you can read from cat /proc/iomem.

Interlace

Answer 5

You can use this operation and it works for me.

vi /ets/default/grub

and change this line to disable i801_smbus feature.

GRUB_CMDLINE_LINUX_DEFAULT="i2c-i801.disable_features=0x10 quiet splash"

Fraxinella

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Alba