FAQs
English (US) 日本語 | Japanese 中文 | Chinese (Simplified)

Conv2D not working with stride=16 for NPU, Kernel crashes

cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Conv2D not working with stride=16 for NPU, Kernel crashes

‎11-08-2023 05:22 AM
1,026 Views
jlamp
Contributor II

Hi,

I have created the next tflite model which I attached,

conv2d_16stride.png

I am trying to execute in the NPU using tflite-vx-delegate but the kernel crashes and the execution stops working.

This is the code

 

 

import tflite_runtime.interpreter as tflite

def eval_model(model, delegates):
    print(f"Evaluating model: {model}")
    interpreter = tflite.Interpreter(model_path=model, experimental_delegates=delegates)
    interpreter.allocate_tensors()
    input_details = interpreter.get_input_details()
    output_details = interpreter.get_output_details()
    input_data = np.random.randn(*input_shape).astype(input_details[0]["dtype"])
    interpreter.set_tensor(input_details[0]['index'], input_data)
interpreter.invoke()

vx_delegate = tflite.load_delegate(library="/usr/lib/libvx_delegate.so", options={"backends": "CpuAcc", "logging-severity":"error"})
delegates = [vx_delegate]

model_paths = ["/model_full_integer_quant.tflite"]

for model in model_paths:
    eval_model(model, delegates)

 

 

 

This is happening when stride > 15.

Used BSP = kirkstone 5.15.71_2.2.0

And here Kernel error

<details><summary>Kernel error</summary>

```
[59301.060254] **************************
[59301.064021] *** GPU DRV CONFIG ***
[59301.067794] **************************
[59301.071560] Galcore version 6.4.3.398061
[59301.075498] Galcore options:
[59301.078390] irqLine = 71
[59301.081975] registerMemBase = 0x38000000
[59301.086252] registerMemSize = 0x00008000
[59301.090531] irqLine2D = 72
[59301.094115] registerMemBase2D = 0x38008000
[59301.098391] registerMemSize2D = 0x00008000
[59301.102668] contiguousSize = 0x10000000
[59301.106947] contiguousBase = 0x0
[59301.110616] externalSize = 0x00000000
[59301.114895] externalBase = 0x0
[59301.118564] bankSize = 0x00000000
[59301.122847] fastClear = -1
[59301.126432] compression = 15
[59301.130017] powerManagement = 1
[59301.133513] baseAddress = 0x40000000
[59301.137791] physSize = 0xC0000000
[59301.142071] recovery = 0
[59301.145568] stuckDump = 0
[59301.149062] userClusterMask = 0x0
[59301.152733] GPU smallBatch = 1
[59301.156225] allMapInOne = 1
[59301.159722] mmuException = 1
[59301.163218] irqs =
[59301.163221] 71,
[59301.166630] 73,
[59301.168476] -1,
[59301.170314] -1,
[59301.172149] -1,
[59301.173997] -1,
[59301.175836] -1,
[59301.177683] -1,
[59301.179522] 72,
[59301.181375] -1,
[59301.183214]
[59301.186551] registerBases =
[59301.186554] 0x38000000,
[59301.189964] 0x38500000,
[59301.192505] 0x00000000,
[59301.195038] 0x00000000,
[59301.197578] 0x00000000,
[59301.200110] 0x00000000,
[59301.202651] 0x00000000,
[59301.205192] 0x00000000,
[59301.207726] 0x38008000,
[59301.210263] 0x00000000,
[59301.212805]
[59301.216833] registerSizes =
[59301.216836] 0x00008000,
[59301.220244] 0x00020000,
[59301.222777] 0x00000000,
[59301.225317] 0x00000000,
[59301.227849] 0x00000000,
[59301.230389] 0x00000000,
[59301.232930] 0x00000000,
[59301.235463] 0x00000000,
[59301.238004] 0x00008000,
[59301.240547] 0x00000000,
[59301.243080]
[59301.247114] chipIDs =
[59301.247117] 0xFFFFFFFF,
[59301.250526] 0xFFFFFFFF,
[59301.253068] 0xFFFFFFFF,
[59301.255603] 0xFFFFFFFF,
[59301.258141] 0xFFFFFFFF,
[59301.260683] 0xFFFFFFFF,
[59301.263217] 0xFFFFFFFF,
[59301.265756] 0xFFFFFFFF,
[59301.268298] 0xFFFFFFFF,
[59301.270832] 0xFFFFFFFF,
[59301.273370]
[59301.277399] core 0 internal sRAMBases =
[59301.277403] 0xFFFFFFFFFFFFFFFF,
[59301.281505] 0xFFFFFFFFFFFFFFFF,
[59301.284742]
[59301.289463] core 1 internal sRAMBases =
[59301.289466] 0xFFFFFFFFFFFFFFFF,
[59301.293573] 0xFFFFFFFFFFFFFFFF,
[59301.296811]
[59301.301533] core 2 internal sRAMBases =
[59301.301536] 0xFFFFFFFFFFFFFFFF,
[59301.305638] 0xFFFFFFFFFFFFFFFF,
[59301.308874]
[59301.313594] core 3 internal sRAMBases =
[59301.313598] 0xFFFFFFFFFFFFFFFF,
[59301.317701] 0xFFFFFFFFFFFFFFFF,
[59301.320937]
[59301.325655] core 4 internal sRAMBases =
[59301.325659] 0xFFFFFFFFFFFFFFFF,
[59301.329761] 0xFFFFFFFFFFFFFFFF,
[59301.332995]
[59301.337718] core 5 internal sRAMBases =
[59301.337721] 0xFFFFFFFFFFFFFFFF,
[59301.341822] 0xFFFFFFFFFFFFFFFF,
[59301.345058]
[59301.349783] core 6 internal sRAMBases =
[59301.349786] 0xFFFFFFFFFFFFFFFF,
[59301.353887] 0xFFFFFFFFFFFFFFFF,
[59301.357124]
[59301.361847] core 7 internal sRAMBases =
[59301.361850] 0xFFFFFFFFFFFFFFFF,
[59301.365954] 0xFFFFFFFFFFFFFFFF,
[59301.369195]
[59301.373917] core 8 internal sRAMBases =
[59301.373920] 0xFFFFFFFFFFFFFFFF,
[59301.378023] 0xFFFFFFFFFFFFFFFF,
[59301.381259]
[59301.385982] core 9 internal sRAMBases =
[59301.385985] 0xFFFFFFFFFFFFFFFF,
[59301.390088] 0xFFFFFFFFFFFFFFFF,
[59301.393324]
[59301.398052] External sRAMBases =
[59301.398055] 0xffffffffffffffff,
[59301.401550] 0xffffffffffffffff,
[59301.404790]
[59301.409514] mmuPageTablePool = 1
[59301.413012] mmuDynamicMap = 1
[59301.416507] Build options:
[59301.419213] gpuTimeout = 30000
[59301.422709] gcdGPU_2D_TIMEOUT = 20000
[59301.426553] gcdINTERRUPT_STATISTIC = 1
[59301.430486] GPU[1](ChipModel=0x8000 ChipRevision=0x8002):
[59301.436183] **************************
[59301.439939] *** GPU STATE DUMP ***
[59301.443701] **************************
[59301.447462] axi = 0x000000DF
[59301.450960] idle = 0x7FFFFFF6
[59301.454457] FE not idle
[59301.457259] SH not idle
[59301.460054] DMA appears to be stuck at this address:
[59301.465197] 0x72238F98
[59301.467904] dmaLow = 0x0804D808
[59301.471400] dmaHigh = 0x00001800
[59301.474899] dmaState = 0x00000905
[59301.478396] command state = 5 (PAR_LOAD1_ST)
[59301.483542] command DMA state = 1 (CMD_START_ST)
[59301.488690] command fetch state = 2 (FET_VALID_ST)
[59301.493846] DMA request state = 0 (REQ_IDLE_ST)
[59301.498908] cal state = 0 (CAL_IDLE_ST)
[59301.503970] VE request state = 0 (VER_IDLE_ST)
[59301.509028] Debug registers:
[59301.512084] RA[0] debug registers:
[59301.515848] [00] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.524302] [08] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.532755] [10] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.541206] [18] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.549655] [20] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.558107] [28] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.566557] [30] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.575007] [38] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.583457] [40] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.591907] [48] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.600357] [50] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.608806] [58] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.617259] [60] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.625709] [68] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.634159] [70] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.642620] [78] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.651081] [80] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.659538] [88] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.667990] [90] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.676440] [98] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.684890] [A0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.693338] [A8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.701786] [B0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.710234] [B8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.718682] [C0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.727130] [C8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.735578] [D0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.744032] [D8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.752482] [E0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.760933] [E8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.769383] [F0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.777835] [F8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.786324] TX[0] debug registers:
[59301.790099] [00] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.798552] [08] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.807005] [10] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.815455] [18] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.823908] [20] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.832357] [28] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.840808] [30] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.849257] [38] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.857706] [40] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.866179] [48] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.874638] [50] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.883087] [58] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.891535] [60] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.899984] [68] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.908442] [70] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.916893] [78] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.925339] FE[0] debug registers:
[59301.929100] [00] 722391B0 11110000 0F0E0706 00000000 000601A4 0028000C FFFFFFFF 00000000
[59301.937551] [08] 0000000C 00000000 00000000 00000000 00000000 AE4407F2 00000050 BABEF00D
[59301.945999] [10] 00000000 3EDB2431 3EDE7C6C 45674567 45674567 45674567 45674567 45674567
[59301.954447] [18] 45674567 45674567 12345678 45674567 12345678 45674567 00000000 00000000
[59301.962899] [20] FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
[59301.971354] [28] FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
[59301.979803] [30] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.988258] [38] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59301.996709] [40] 00000400 45674567 45674567 45674567 45674567 45674567 45674567 45674567
[59302.005157] [48] 45674567 45674567 45674567 45674567 45674567 45674567 45674567 45674567
[59302.013606] [50] 00000000 00000000 00000000 00000000 00000000 45674567 45674567 45674567
[59302.022066] [58] 45674567 45674567 45674567 45674567 45674567 45674567 45674567 45674567
[59302.030520] [60] 45674567 45674567 45674567 45674567 45674567 45674567 45674567 45674567
[59302.038974] [68] 45674567 45674567 45674567 45674567 45674567 45674567 45674567 45674567
[59302.047424] [70] 45674567 45674567 45674567 45674567 45674567 45674567 45674567 45674567
[59302.055875] [78] 45674567 45674567 45674567 45674567 45674567 45674567 45674567 45674567
[59302.064325] [80] 722391B0 11110000 0F0E0706 00000000 000601A4 0028000C FFFFFFFF 00000000
[59302.072775] [88] 0000000C 00000000 00000000 00000000 00000000 AE4407F2 00000051 BABEF00D
[59302.081224] [90] 00000000 46EA8461 46EDDC9D 45674567 45674567 45674567 45674567 45674567
[59302.089679] [98] 45674567 45674567 12345678 45674567 12345678 45674567 00000000 00000000
[59302.098133] [A0] FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
[59302.106589] [A8] FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
[59302.115043] [B0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.123492] [B8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.131941] [C0] 00000400 45674567 45674567 45674567 45674567 45674567 45674567 45674567
[59302.140392] [C8] 45674567 45674567 45674567 45674567 45674567 45674567 45674567 45674567
[59302.148848] [D0] 00000000 00000000 00000000 00000000 00000000 45674567 45674567 45674567
[59302.157301] [D8] 45674567 45674567 45674567 45674567 45674567 45674567 45674567 45674567
[59302.165751] [E0] 45674567 45674567 45674567 45674567 45674567 45674567 45674567 45674567
[59302.174201] [E8] 45674567 45674567 45674567 45674567 45674567 45674567 45674567 45674567
[59302.182649] [F0] 45674567 45674567 45674567 45674567 45674567 45674567 45674567 45674567
[59302.191098] [F8] 45674567 45674567 45674567 45674567 45674567 45674567 45674567 45674567
[59302.199544] PE[0] debug registers:
[59302.203304] [00] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.211753] [08] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.220201] [10] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.228650] [18] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.237107] [20] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.245556] [28] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.254005] [30] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.262453] [38] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.270903] [40] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.279356] [48] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.287809] [50] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.296262] [58] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.304711] [60] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.313159] [68] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.321608] [70] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.330057] [78] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.338506] [80] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.346953] [88] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.355401] [90] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.363852] [98] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.372302] [A0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.380752] [A8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.389201] [B0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.397650] [B8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.406104] [C0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.414555] [C8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.423006] [D0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.431454] [D8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.439903] [E0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.448358] [E8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.456813] [F0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.465263] [F8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.473711] PE[1] debug registers:
[59302.477472] [00] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.485953] [08] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.494454] [10] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.502924] [18] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.511387] [20] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.519839] [28] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.528289] [30] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.536743] [38] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.545196] [40] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.553648] [48] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.562099] [50] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.570550] [58] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.578999] [60] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.587447] [68] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.595895] [70] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.604344] [78] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.612797] [80] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.621247] [88] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.629696] [90] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.638145] [98] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.646595] [A0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.655045] [A8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.663499] [B0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.671950] [B8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.680403] [C0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.688853] [C8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.697309] [D0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.705764] [D8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.714214] [E0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.722664] [E8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.731113] [F0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.739567] [F8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.748014] DE[0] debug registers:
[59302.751793] [00] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.760256] [08] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.768741] [10] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.777198] [18] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.785685] [20] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.794145] [28] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.802600] [30] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.811050] [38] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.819504] [40] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.827956] [48] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.836407] [50] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.844858] [58] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.853309] [60] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.861786] [68] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.870244] [70] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.878696] [78] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.887146] [80] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.895595] [88] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.904045] [90] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.912496] [98] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.920952] [A0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.929415] [A8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.937870] [B0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.946320] [B8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.954770] [C0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.963218] [C8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.971667] [D0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.980116] [D8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.988568] [E0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59302.997018] [E8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.005467] [F0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.013914] [F8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.022361] SH[0] debug registers:
[59303.026123] [00] 32FBEAAC 951FC000 0010F004 00000A9A 7F41E344 80000000 80000000 00012180
[59303.034571] [08] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.043021] [10] 00000000 0003114C 00000000 00000000 000009CA 00000000 8040D778 8040D895
[59303.051476] [18] 80C4DB5F 00000000 80C1D619 CAFEF00D CAFEF00D CAFEF00D 00000000 DD000103
[59303.059927] [20] 00000000 00000000 00000000 00000100 00000000 2697F97F 00000000 00000000
[59303.068378] [28] 00000000 00000010 00000000 00000000 00000000 00000000 00000000 57A00011
[59303.076827] [30] FFFFEFFE BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.085276] [38] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.093726] [40] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.102174] [48] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.110621] [50] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.119075] [58] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.127525] [60] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.135974] [68] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.144423] [70] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.152873] [78] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.161323] [80] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.169777] [88] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.178228] [90] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.186677] [98] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.195124] [A0] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.203574] [A8] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.212025] [B0] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.220502] [B8] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.228962] [C0] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.237413] [C8] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.245868] [D0] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.254318] [D8] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.262768] [E0] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.271216] [E8] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.279665] [F0] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D
[59303.288117] [F8] BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D BAB4F00D DEADBEEF
[59303.296564] PA[0] debug registers:
[59303.300326] [00] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.308775] [08] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.317223] [10] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.325671] [18] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.334121] [20] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.342574] [28] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.351026] [30] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.359475] [38] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.367925] [40] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.376377] [48] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.384828] [50] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.393278] [58] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.401727] [60] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.410175] [68] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.418625] [70] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.427075] [78] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.435524] [80] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.443977] [88] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.452431] [90] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.460883] [98] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.469337] [A0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.477787] [A8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.486237] [B0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.494689] [B8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.503146] [C0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.511598] [C8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.520047] [D0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.528496] [D8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.536945] [E0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.545394] [E8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.553843] [F0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.562291] [F8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.570739] SE[0] debug registers:
[59303.574500] [00] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.582949] [08] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.591399] [10] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.599853] [18] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.608305] [20] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.616755] [28] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.625209] [30] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.633659] [38] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.642108] [40] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.650557] [48] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.659003] [50] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.667452] [58] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.675900] [60] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.684350] [68] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.692802] [70] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.701277] [78] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.709736] [80] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.718187] [88] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.726635] [90] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.735084] [98] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.743533] [A0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.751985] [A8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.760434] [B0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.768883] [B8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.777332] [C0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.785794] [C8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.794248] [D0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.802698] [D8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.811147] [E0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.819597] [E8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.828053] [F0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.836507] [F8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.844956] MC[0] debug registers:
[59303.848719] [00] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.857168] [08] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.865632] [10] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.874091] [18] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59303.882545] [20] 00000000 00000000 12345678 12345678 12345678 12345678 12345678 12345678
[59303.890997] [28] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59303.899447] [30] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59303.907897] [38] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59303.916348] [40] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59303.924795] [48] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59303.933246] [50] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59303.941696] [58] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59303.950149] [60] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59303.958599] [68] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59303.967047] [70] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59303.975495] [78] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59303.983948] [80] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59303.992398] [88] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.000851] [90] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.009301] [98] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.017750] [A0] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.026199] [A8] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.034648] [B0] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.043097] [B8] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.051546] [C0] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.059996] [C8] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.068448] [D0] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.076899] [D8] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.085348] [E0] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.093795] [E8] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.102244] [F0] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.110692] [F8] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.119138] MC[1] debug registers:
[59304.122900] [00] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.131355] [08] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.139805] [10] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.148253] [18] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.156701] [20] 00000000 00000000 12345678 12345678 12345678 12345678 12345678 12345678
[59304.165150] [28] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.173598] [30] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.182048] [38] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.190504] [40] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.198955] [48] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.207404] [50] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.215854] [58] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.224329] [60] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.232829] [68] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.241314] [70] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.249799] [78] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.258291] [80] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.266809] [88] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.275304] [90] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.283792] [98] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.292280] [A0] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.300757] [A8] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.309227] [B0] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.317700] [B8] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.326177] [C0] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.334658] [C8] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.343143] [D0] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.351647] [D8] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.360139] [E0] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.368617] [E8] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.377088] [F0] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.385566] [F8] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59304.394047] HI[0] debug registers:
[59304.397841] [00] 00000018 00000040 00000061 00000000 AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.406333] [08] 0000093C 00000002 00000002 AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.414809] [10] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.423274] [18] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.431735] [20] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.440187] [28] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.448642] [30] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.457093] [38] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.465546] [40] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.473998] [48] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.482456] [50] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.490916] [58] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.499375] [60] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.507828] [68] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.516284] [70] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.524740] [78] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.533191] [80] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.541641] [88] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.550092] [90] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.558542] [98] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.566991] [A0] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.575442] [A8] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.583891] [B0] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.592340] [B8] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.600790] [C0] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.609245] [C8] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.617697] [D0] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.626146] [D8] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.634595] [E0] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.643047] [E8] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.651504] [F0] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.659965] [F8] AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
[59304.668418] USC[1] debug registers:
[59304.672258] [C0] 11500000 DC002F2F 000015C5 00000000 00005714 00000000 00002E2B 0000184B
[59304.680708] [C8] 00002B1C 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.689160] [D0] 00000CAD 00000000 000032B4 00000000 00000CAD 00000000 000032B4 00000000
[59304.697612] [D8] 00000CAD 00000000 000032B4 00000000 00000CAD 00000000 000032B4 00000000
[59304.706065] [E0] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59304.714516] [E8] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59304.722971] [F0] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59304.731425] [F8] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59304.739873] L2[0] debug registers:
[59304.743633] [00] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.752085] [08] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.760535] [10] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.768990] [18] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.777441] [20] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.785899] [28] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.794355] [30] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.802806] [38] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.811257] [40] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.819707] [48] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.828159] [50] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.836615] [58] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.845067] [60] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.853516] [68] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.861978] [70] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.870443] [78] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.878899] [80] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.887354] [88] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.895809] [90] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.904291] [98] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.912759] [A0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.921213] [A8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.929666] [B0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.938116] [B8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.946568] [C0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.955018] [C8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.963470] [D0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.971921] [D8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.980378] [E0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.988831] [E8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59304.997288] [F0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59305.005737] [F8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59305.014187] NN[1] debug registers:
[59305.017954] [00] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59305.026415] [08] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59305.034871] [10] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59305.043325] [18] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59305.051778] [20] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59305.060231] [28] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59305.068683] [30] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59305.077135] [38] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59305.085588] [40] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59305.094041] [48] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59305.102492] [50] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59305.110946] [58] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59305.119400] [60] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59305.127855] [68] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59305.136311] [70] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59305.144770] [78] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59305.153225] [80] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59305.161680] [88] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59305.170134] [90] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59305.178603] [98] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59305.187062] [A0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59305.195519] [A8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59305.203974] [B0] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59305.212428] [B8] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59305.220879] [C0] 11500000 DC002F2F 000015C5 00000000 00005714 00000000 00002E2B 0000184B
[59305.229331] [C8] 00002B1C 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59305.237784] [D0] 00000CAD 00000000 000032B4 00000000 00000CAD 00000000 000032B4 00000000
[59305.246240] [D8] 00000CAD 00000000 000032B4 00000000 00000CAD 00000000 000032B4 00000000
[59305.254693] [E0] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59305.263147] [E8] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59305.271601] [F0] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59305.280053] [F8] 00000004 00000004 00000004 00000004 00000004 00000004 00000004 00000004
[59305.288497] Other Registers[0]:
[59305.291986] [0x0040] 0x00003728
[59305.295657] [0x0044] 0x000024F0
[59305.299330] [0x004C] 0x000024F0
[59305.303007] [0x0050] 0x0000093C
[59305.306697] [0x0054] 0x0000093C
[59305.310373] [0x0058] 0x00003728
[59305.314047] [0x005C] 0x00000E44
[59305.317717] [0x0060] 0x00000E44
[59305.321389] [0x043C] 0x00000000
[59305.325057] [0x0440] 0x00000000
[59305.328728] [0x0444] 0x00000000
[59305.332399] [0x0414] 0x3C000000
[59305.336060] [0x0100] 0x00140021
[59305.339732] MMU status from MC[0]:
[59305.343489] GPU[1](ChipModel=0x8000 ChipRevision=0x8002):
[59305.348895] **************************
[59305.352652] *** MMU ERROR DUMP ***
[59305.356409] **************************
[59305.360158] MMU status = 0x00000000
[59305.363832] Link stack:
[59305.366648] [0x00] 0xAE4407F2 [0x01] 0xAE4407F2 [0x02] 0xAE4407F2 [0x03] 0xAE4407F2
[59305.374834] [0x04] 0xAE4407F2 [0x05] 0xAE4407F2 [0x06] 0xAE4407F2 [0x07] 0xAE4407F2
[59305.383020] [0x08] 0xAE4407F2 [0x09] 0xAE4407F2 [0x0A] 0xAE4407F2 [0x0B] 0xAE4407F2
[59305.391205] [0x0C] 0xAE4407F2 [0x0D] 0xAE4407F2 [0x0E] 0xAE44711C [0x0F] 0x8E4407F2
[59305.399419] PRE_STACK:
[59305.401964] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59305.409977] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59305.417991] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59305.426008] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59305.434022] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59305.442034] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59305.450052] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59305.458068] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59305.466114] CMD_STACK:
[59305.468658] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59305.476670] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59305.484686] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59305.492701] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59305.500719] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59305.508734] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59305.516746] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59305.524763] 12345678 12345678 12345678 12345678 12345678 12345678 12345678 12345678
[59305.532774] LINK_STACK:
[59305.535393] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59305.543405] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59305.551423] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59305.559437] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59305.567450] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59305.575463] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59305.583477] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59305.591490] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[59305.599503] **************************
[59305.603260] ***** SW COUNTERS *****
[59305.607017] **************************
[59305.610776] Execute Count = 0x0000015B
[59305.614881] Execute Addr = 0x72203F98
[59305.618985] End Addr = 0x72203F88
[59305.623092] CPU: 3 PID: 11253 Comm: kworker/u8:3 Tainted: G C 5.15.71+g7f3699e2238e #1
[59305.632311] Hardware name: SolidRun i.MX8MP HummingBoard Pulse (DT)
[59305.638578] Workqueue: galcore workqueue _TimerFunction
[59305.643813] Call trace:
[59305.646255] dump_backtrace+0x0/0x19c
[59305.649924] show_stack+0x18/0x70
[59305.653238] dump_stack_lvl+0x68/0x84
[59305.656905] dump_stack+0x18/0x34
[59305.660219] gckOS_DumpCallStack+0x10/0x20
[59305.664320] gckHARDWARE_DumpGPUState+0x728/0xfb4
[59305.669025] gckKERNEL_Recovery+0xbc/0x2e4
[59305.673125] _MonitorTimerFunction+0x1f0/0x36c
[59305.677568] _TimerFunction+0x14/0x20
[59305.681229] process_one_work+0x1d0/0x354
[59305.685241] worker_thread+0x13c/0x470
[59305.688993] kthread+0x150/0x160
[59305.692223] ret_from_fork+0x10/0x20
[59305.695823] **************************
[59305.699596] *** EVENT STATE DUMP ***
[59305.703365] **************************
[59305.707126] Unsumbitted Event:
[59305.710363] Untriggered Event:
[59305.713600] [0]:
[59305.715787] [1]:
[59305.717979] [2]:
[59305.720165] [3]:
[59305.722361] [4]:
[59305.724556] [5]:
[59305.726741] [6]:
[59305.728934] gcvHAL_COMMIT_DONE
[59305.732518] [7]:
[59305.734704] gcvHAL_SIGNAL process=11270 signal=0x12d1
[59305.740283] [8]:
[59305.742477] [9]:
[59305.744674] [10]:
[59305.746945] [11]:
[59305.749228] [12]:
[59305.751500] [13]:
[59305.753783] [14]:
[59305.756056] [15]:
[59305.758338] [16]:
[59305.760620] [17]:
[59305.762890] [18]:
[59305.765169] [19]:
[59305.767441] [20]:
[59305.769723] [21]:
[59305.771995] [22]:
[59305.774276] [23]:
[59305.776558] [24]:
[59305.778831] [25]:
[59305.781109] [26]:
[59305.783380] [27]:
[59305.785675] [28]:
[59305.787953] Number of Pending Interrupt: 2
[59305.792243] INTR_ACKNOWLEDGE=0x0
[59305.795656] Notify State=-1
[59305.798633] Pending=0x0
[59305.801263] **************************
[59305.805019] *** PROCESS DB DUMP ***
[59305.808778] **************************
[59305.812537] PID NAME
[59305.815074] 11270 python3
[59305.817880] **************************
[59305.821643] **** COMMAND BUF DUMP ****
[59305.825401] **************************
[59305.829161] Submitted commit stamp = 119
[59305.833266] Executed commit stamp = 118
[59305.837372] DMA Address 0x72238F98
[59305.840901] Command buffer around 0x72238F98:
[59305.845275] Unable to handle kernel paging request at virtual address ffff80010b039000
[59305.853209] Mem abort info:
[59305.856008] ESR = 0x96000005
[59305.859068] EC = 0x25: DABT (current EL), IL = 32 bits
[59305.864420] SET = 0, FnV = 0
[59305.867485] EA = 0, S1PTW = 0
[59305.870640] FSC = 0x05: level 1 translation fault
[59305.875523] Data abort info:
[59305.878408] ISV = 0, ISS = 0x00000005
[59305.882246] CM = 0, WnR = 0
[59305.885217] swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000000041a0d000
[59305.891921] [ffff80010b039000] pgd=10000000fffff003, p4d=10000000fffff003, pud=0000000000000000
[59305.900633] Internal error: Oops: 96000005 [#1] PREEMPT SMP
[59305.906205] Modules linked in: algif_hash algif_skcipher af_alg fsl_jr_uio caam_jr caamkeyblob_desc caamhash_desc caamalg_desc crypto_engine authenc libdes snd_soc_hdmi_codec snd_soc_fsl_easrc adin snd_soc_fsl_sai snd_soc_fsl_asrc snd_soc_simple_card imx_pcm_dma snd_soc_simple_e
[59305.955357] CPU: 3 PID: 11253 Comm: kworker/u8:3 Tainted: G C 5.15.71+g7f3699e2238e #1
[59305.964577] Hardware name: SolidRun i.MX8MP HummingBoard Pulse (DT)
[59305.970844] Workqueue: galcore workqueue _TimerFunction
[59305.976079] pstate: a0000005 (NzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[59305.983042] pc : _DumpBuffer+0x48/0x254
[59305.986883] lr : gckCOMMAND_DumpExecutingBuffer+0x340/0x370
[59305.992458] sp : ffff80001b9a3bd0
[59305.995770] x29: ffff80001b9a3be0 x28: 0000000000000000 x27: 0000000000000000
[59306.002913] x26: ffff8000099fa1f0 x25: 0000000000000080 x24: 0000000072238000
[59306.010053] x23: 00000000671ff000 x22: ffff800009506410 x21: 0000000000000000
[59306.017191] x20: ffff80010b03a000 x19: ffff80010b039000 x18: ffffffffffffffff
[59306.024331] x17: 2020202020202043 x16: 2020202020202020 x15: ffff80009b9a3747
[59306.031470] x14: 0000000000000000 x13: ffff8000098b1cf8 x12: 0000000000000f45
[59306.038609] x11: 0000000000000517 x10: ffff8000098b1cf8 x9 : ffff8000098b1cf8
[59306.045750] x8 : 00000000ffffefff x7 : ffff800009909cf8 x6 : ffff800009909cf8
[59306.052889] x5 : 0000000000001000 x4 : 00000000fffff740 x3 : 00000000000006d8
[59306.060030] x2 : 0000000000001000 x1 : 0000000072238000 x0 : ffff80010b039000
[59306.067169] Call trace:
[59306.069617] _DumpBuffer+0x48/0x254
[59306.073111] gckCOMMAND_DumpExecutingBuffer+0x340/0x370
[59306.078336] gckKERNEL_Recovery+0xd8/0x2e4
[59306.082433] _MonitorTimerFunction+0x1f0/0x36c
[59306.086881] _TimerFunction+0x14/0x20
[59306.090543] process_one_work+0x1d0/0x354
[59306.094556] worker_thread+0x13c/0x470
[59306.098308] kthread+0x150/0x160
[59306.101538] ret_from_fork+0x10/0x20
[59306.105118] Code: 8b191414 4b000037 911042d6 d503201f (29400e62)
[59306.111211] ---[ end trace a133263ff8cfecd1 ]---

Message from syslogd@imx8mpsolidrun at Nov 8 07:58:45 ...
kernel:[59305.900633] Internal error: Oops: 96000005 [#1] PREEMPT SMP

Message from syslogd@imx8mpsolidrun at Nov 8 07:58:45 ...
kernel:[59306.105118] Code: 8b191414 4b000037 911042d6 d503201f (29400e62)
[59570.729393] systemd-journald[206]: Data hash table of /var/log/journal/97039774429b424fb9a3d41f4fc7a92d/system.journal has a fill level at 75.0 (8533 of 11377 items, 6553600 file size, 768 bytes per hash table item), suggesting rotation.
[59570.750508] systemd-journald[206]: /var/log/journal/97039774429b424fb9a3d41f4fc7a92d/system.journal: Journal header limits reached or header out-of-date, rotating.

```

</details>

 

Any idea of why this is happening?

model_full_integer_quant.zip
Reply
4 Replies

‎11-16-2023 09:42 PM
942 Views
brian14
NXP TechSupport
NXP TechSupport

Hi @jlamp, 

I have reviewed your case, and I can confirm that there is an issue executing your Conv2D operator on NPU. It seems that there is a limitation with the parameters of the operator.

I will review in more detail, and I will contact you as soon as possible.

Have a great day!

0 Kudos
Reply

‎11-17-2023 02:42 AM
934 Views
jlamp
Contributor II

Hi @brian14 ,

Thank you for reviewing this, waiting for your news.

Regards!

0 Kudos
Reply

‎11-24-2023 05:24 AM
848 Views
jlamp
Contributor II

Hi @brian14 

 

I am checking a little bit on how this works under the hood, compiling tim-vx, ovxlib, OpenVX and checking galcore kernel module and I am seeing that the code gets blocked in  [`status = vxProcessGraph( graph->g );`](https://github.com/nxp-imx/nn-imx/blob/imx_1.3.0/ovxlib/src/vsi_nn_graph.c#L840) when running the graph `vsi_nn_RunGraph`.

From [OpenVX-api-docs](https://github.com/KhronosGroup/OpenVX-api-docs/blob/openvx_1.3/include/VX/vx_api.h), this call blocks until graph is completed.

But can you tell me who implements this `vxProcessGraph` function? Is this inside galcore driver? Does the NPU use galcore driver?

Thank you very much,

 

Jorge

0 Kudos
Reply

‎12-01-2023 07:25 AM
801 Views
brian14
NXP TechSupport
NXP TechSupport

Hi @jlamp, 

Based on our tests and your experience it seems that the limitation is with the stride >15. I recommend implementing your model limiting this parameter below 15.

Have a great day!

0 Kudos
Reply