Seldon Core

Seldon Core
Name	Seldon Core
Developer	Seldon (company)
Released	2018
Programming language	Python, Go
Operating system	Linux, Microsoft Windows, macOS
License	Apache License

Seldon Core Seldon Core is an open-source platform for deploying, scaling, and managing machine learning models in production. It integrates with cloud-native infrastructures and supports multiple model frameworks, providing tools for model serving, monitoring, routing, and explainability. Seldon Core is widely used alongside platforms and projects in the Kubernetes, Istio (service mesh), and Prometheus ecosystems.

Overview

Seldon Core originated as a project by Seldon and has been adopted in contexts involving Amazon Web Services, Google Cloud Platform, Microsoft Azure, Red Hat OpenShift, and IBM Cloud. It targets workflows that include model packaging from TensorFlow, PyTorch, scikit-learn, XGBoost, and ONNX runtimes, and integrates with orchestration tools such as Argo and Kubeflow. The project is part of cloud-native conversations alongside Cloud Native Computing Foundation, CNCF Landscape, and related projects like Knative and Linkerd.

Architecture

Seldon Core's architecture emphasizes microservice patterns used in Kubernetes clusters, leveraging sidecar proxies similar to Envoy (software), and service meshes like Istio (service mesh) or Linkerd. Model servers are packaged as containers following practices from Docker, and deployments are managed with Helm (software). The architecture supports canary deployments inspired by techniques used in Spinnaker and integrates with CI/CD systems such as Jenkins, GitLab CI/CD, and CircleCI. Observability is provided through integration with Prometheus, Grafana, Elastic Stack, and tracing systems including Jaeger (software) and Zipkin.

Deployment and Integration

Seldon Core deploys models using Kubernetes custom resources and operators, aligning with patterns from Operator Framework and Kubernetes Operators. It connects to cloud-native identity systems like Keycloak and Dex for authentication and can be orchestrated within platforms such as Red Hat OpenShift and Google Kubernetes Engine. Integration pipelines commonly use Argo CD, Tekton, and Flux for GitOps workflows. For inference at the edge, Seldon Core can interoperate with projects like K3s and KubeEdge.

Features and Components

Seldon Core includes model serving components compatible with TensorFlow Serving, ONNX Runtime, TorchServe, and custom container inference servers. It exposes REST and gRPC endpoints and supports transformation chains similar to patterns in Kubeflow Pipelines. Feature sets encompass request/response logging to Fluentd or Filebeat, metrics export to Prometheus, and dashboarding via Grafana. It provides explainability hooks that integrate with tools and research like SHAP and LIME and supports A/B testing and traffic splitting comparable to features in Istio (service mesh) and Linkerd. Packaging and model versioning align with standards used by MLflow, DVC, and Model Asset Exchange.

Use Cases and Applications

Organizations deploy Seldon Core for use cases in finance with firms akin to Goldman Sachs, JPMorgan Chase, and HSBC for fraud detection and risk scoring, in healthcare alongside institutions like Mayo Clinic and NHS for diagnostic pipelines, and in retail with companies similar to Amazon, Walmart, and Shopify for recommendation systems. It is used by research groups associated with University of Cambridge, University of Oxford, Stanford University, and Massachusetts Institute of Technology for reproducible inference experiments. Seldon Core supports real-time inference in telecommunications by organizations comparable to Verizon and Vodafone and in autonomous systems similar to Waymo and Tesla, Inc. for sensor fusion serving.

Security and Compliance

Seldon Core supports TLS termination and mutual TLS via integrations with Istio (service mesh), Envoy (software), and cert-manager. It can leverage role-based access control from Kubernetes and integrate audit logging with Elastic Stack and Splunk. For compliance, deployments are tailored to standards relevant to HIPAA, GDPR, and SOC 2 processes, and can be incorporated into governance frameworks used by enterprises like Oracle Corporation and SAP SE. Secrets management commonly integrates with HashiCorp Vault, AWS Secrets Manager, and Azure Key Vault.

Performance and Scalability

Seldon Core scales horizontally on Kubernetes using autoscaling features from Kubernetes Horizontal Pod Autoscaler and custom metrics via Prometheus Adapter. Load balancing leverages Envoy (software), NGINX, and cloud load balancers from Amazon Web Services, Google Cloud Platform, and Microsoft Azure. GPU acceleration support is provided through NVIDIA drivers and runtimes like NVIDIA CUDA and orchestration for GPUs similar to Kubeflow patterns. Benchmarks and performance tests often reference tooling such as Locust (software), k6 (software), and JMeter to assess latency and throughput.

Category:Machine learning software Category:Kubernetes