Reported impact
Business outcome
WeTech replaced a heavier self-managed training approach with a managed lakehouse training stack that could scale with the business. The new platform delivered more than 3x engine performance versus the open-source baseline, second-level resource provisioning, and a complete migration of all big-data compute tasks to the cloud in roughly one month, while reducing overall storage and operations costs.
- Cost Reduction
- Productivity Improvement
- Scalability
Solution
WeTech standardized its machine-learning platform on Alibaba Cloud EMR Serverless Spark and OSS. Instead of managing long-lived compute clusters, the team uses the serverless Spark service as a unified execution layer for data processing, analysis, and machine-learning jobs, taking advantage of the Fusion engine, vectorized execution, and RSS enhancements to improve performance over open-source Spark. The platform is tightly integrated with the broader lakehouse environment: dw-shell handles data collection, Apache Paimon provides the lake storage format, and Spark, Flink, and Hive are combined to form a full data-lake ecosystem for training and monitoring scenarios. By using compute-storage separation with OSS and task-level elastic provisioning, WeTech can run large-scale risk-model training with much lower infrastructure overhead while still giving data scientists familiar Python, SQL, PySpark, notebook, and scheduling workflows.
Data flow
Collected lending and user-behavior data is gathered through WeTech's ingestion tooling into the lakehouse, stored through OSS and Paimon, and then processed by EMR Serverless Spark for feature preparation, model training, and analytical jobs. The resulting models and datasets are used by WeTech's financial risk-evaluation workflows and related real-time monitoring or analysis scenarios.