Got same error with 3.0.35-2666-gbdde708 built by LTIB on iMX6Q SABRE-SD.
Checked YOCTO's builds v1.4 (silex works) and v1.5 (same error -5). BTW - YOCTO let you choose between FSL's or kernel's tree wifi driver package. YOCTO1.4 just took FSL's package. Firmware (2.1.1) taken from v1.4 - started OK on LTIB's build except errors -22 or smth (dont remember - at least I saw local APs by iwlist wlan0 scan).
https://community.freescale.com/thread/311701 points to the patch to fix iMX6Q errata concerning SDv3 ioport.
I failed to apply the patch (http://www.spinics.net/lists/arm-kernel/msg271614.html) due to big differences in the sdhci driver's sources.
Instead, compat-wireless driver package v3.6.8-1 works OK (with any of three silex firmware packs I got) with Silex SX-SDCAN on mentioned YOCTO's and LTIB's builds: http://wireless.kernel.org/en/users/Download/stable/#Building_and_installing
Just remove mac80211 and Atheros drivers from kernel and set cfg80211 to be compiled as M. Compile the backport using guidelines from https://community.freescale.com/thread/295877
As I failed to get "make install" to work - just load compat's generated modules in certain order: compat.ko, cfg80211.ko, mac80211.ko, ath.ko, ath6kl_core.ko, ath6kl_sdio.ko.
The issue you might face loading ath6kl_core.ko - unresolved "simple_open". The fix is to have .configure file (in compat folder) exporting
export CONFIG_COMPAT_KERNEL_3_1=y
export CONFIG_COMPAT_KERNEL_3_2=y
export CONFIG_COMPAT_KERNEL_3_3=y
export CONFIG_COMPAT_KERNEL_3_4=y
export CONFIG_COMPAT_KERNEL_3_5=y
export CONFIG_COMPAT_KERNEL_3_6=y
export CONFIG_COMPAT_KERNEL_3_7=y
export CONFIG_COMPAT_NET_SCH_CODEL=m
export CONFIG_COMPAT_NET_SCH_FQ_CODEL=m
Also I removed CONFIG_PCI and CONFIG_ATH5K exports from config.mk - to get rid from kernel related conflicts/deps etc.
Best Regards!