Caching of more than 5 binary graphs fails

[Traubsi]

Hi, 

I'm suing the i.MX 8 and use tensorflow and the lib_vx_delegate to run ML models on NPU.

To get rid of the warmup-time I applied cashing as stated here:i.MX 8M Plus NPU Warmup Time (mouser.com). Therefore I set the environment variables VIV_VX_CACHE_BINARY_GRAPH_DIR=`pwd`
and VIV_VX_ENABLE_CACHE_GRAPH_BINARY="1". Caching works and I see the saved binary graphs (.nb files).

When storing more than 5 models, the oldest binary graph gets deleted. Do you have any idea why I cannot use caching with more than 5 models?

[joanvicient]

Hello,

We are using 9 different models and the warm-up time without the cache is too long.

Can we get any issue if, as suggested by Traubsi, we store the cache binaries in a different directory, and we load models one by one using symlinks (also creating them one by one and removing them before starting loading the next model) so there is never reached this max number of models?

Is it planned to fix it so more than 5 models can be cached automatically?

[Traubsi]

Thank you for this quick response. 

We managed it to store the existing binary graphs somewhere else and create symlinks. Do you have other ideas?

Is there a reason why the software just works with 5 models?

Kind regards

[Bio_TICFSL]

Hello,

You may need to store in different part the previous binary because yes the software just work with 5 models.

regards