Loading a pyspark ML model in a non-Spark environment

rohitbhat_intel — Fri, 14 Feb 2020 06:22:08 GMT

I am interested in deploying a machine learning model in python, so predictions can be made through requests to a server.

This was suggested by a friend who did the Machine Learning course to create a Cloudera cluster and take advantage of Spark to develop the models, by using the library pyspark. I would like to know how the model can be saved in order to employ it on the server.

I have seen that the different algorithms have the .save functions (like it is answered in this post How to save and load MLLib model in Apache Spark), but as the server will be in a different machine without Spark and not in the Cloudera cluster, I don't know if it is possible to use their .load and .predict functions.

Can it be made by using the pyspark library functions for prediction without Spark underneath? Or would I have to do any transformations in order to save the model and use it elsewhere?

topic Loading a pyspark ML model in a non-Spark environment in eIQ Machine Learning Software

Loading a pyspark ML model in a non-Spark environment