How to set-up a ML SaaS/PaaS Product Team

Recently I got asked this question by a C-level contact of mine, “How to set-up a ML SaaS/PaaS Product Team? What kind of profiles, what kind of team structures do I need to set up this ML product team from scratch?”

In this write-up I have depicted an initial version of the roles / responsibilities of a bare minimum ML Product Development team. This is very much influenced by the current best practices and lifecycles of developing a typical a data science product to best of my understanding — as shown in the diagram below.

An ML product development has a lot common with typical software Product Management and Engineering. ML Platform engineering is also quite different from typical software product in a number of ways. Here are two important ones that comes to me immediately:

1. You would think of addition of ML to your product as a major investment. It is a product by its own right. You can not subordinate an ML journey under your Analytics team leader, for example.
2. You would have a fully functional and dedicated ML Product Management role. This introduces a fundamental shift in thinking about the product value, its roadmap and all that.

Also the team may have emphasis on certain special areas of other technologies based on the architecture of your product: For example, when your product deals with real-time use cases such as infrastructure monitoring, IoT, fleet management etc. you are dealing with pretty sophisticated streaming-data engineering. Also as you are building a product and not just once use case, you need to consider how to make the services, APIs, and documentations available for your customers — sometimes other ML engineers and developers who would use your platform. With that context in mind — here are the roles/responsibilities.

ML Product Manager

Leads the product management aspects such as product vision, roadmap etc.

Expertise

• 10+ years of demonstrated technology product management experience in the field of ML and analytics
• Deep knowledge of AI/ML software technologies such as cutting edge technologies, (especially open source projects) and new technological and innovation in the AI/Ml areas
• Hands-on expertise of implementing relevant AI/ML use cases
• Expertise in defining and communicating how an ML product brings about disruptive value in terms of measurable business KPIs
• Must understand the business needs and trends of target customer market(s) and industries
• BS or higher in Computer Science (or equivalent)

Responsibilities

• Own and drive product vision, strategy, roadmap down to specifications such as product features, technical requirements
• Partners closely with engineering teams to be accountable for the product development life cycle e.g., design, coding, integration, testing, documentation and release cycles of the product
• Liaise with the UX team for alignment with the UX strategy
• Design user friendly APIs for 3rd-party app developers and customers
• Own the product backlog and prioritise product features & bug-fixes to define release payloads
• Responsible for program managing the complete product lifecycle including planning, scheduling, cost management, risk management
• Liaise with the sales and marketing teams to support GTM activities with literatures, demonstrations and prototypes
• Liaise with support teams, pre-sales teams & customers to understand product feedback and feature requests

ML Engineering Manager

Leads the engineering effort to build and ship the product.

Expertise

• 15+ years hands-on experience and knowledge on development life cycle of distributed systems, data and AI platform and products, technological alternatives, and architecture methodologies for multi-platform environments
• Experience with ML and deep learning frameworks, such as pandas, Torch, Caffe, TensorFlow, mxnet, scikit-learn, numpy etc.
• Strong experience with streaming data management at scale
• Strong working knowledge of ML algorithms including decision trees, probability networks, association rules, clustering, regression, and neural networks.
• Executive-level communication and interpersonal skills
• Experience of leading cross-functional engineering teams
• Experience of complex stakeholder management and building organisational alignment
• BS or higher in Computer Science (or equivalent)

Responsibilities

• Hire & coach a team of high-impact engineers to ship industry-leading AI/ML products
• Define and drive the technology strategy and product architecture
• Be the accountable leader for software engineering processes, automation and tools
• Be the accountable leader for building, managing and scaling the ML engineering infrastructure
• Be the accountable leader for product quality including security, scalability, availability, maintainability and operational ease
• Be the accountable leader for quality product documentation and timely updates to product documentation aligned with the release cadence
• Have the technical capacity to partner with tech leads and be comfortable in diving deep into operations issues/ incidents

Data Engineer

Builds and ships the data-pipelining part of the product

Expertise

• 5+ years of hands-on expertise in implementing modern large scale data management technologies
• 5+ years of hands-on expertise in both batch and streaming data processing pipelines
• 5+ years of hands-on expertise in building scale-out, SaaS/PaaS based secure data infrastructure
• 5+ years of hands-on expertise in Spark, Kubernetes and container technologies
• Additional experience with data visualisation, data analytics, and data mining
• Understanding of various data security, data residency, data privacy and regulatory compliance requirements around data management
• BS or higher in Computer Science (or equivalent)

Responsibilities

• Design the data platform architecture
• Build the data platform for enabling data collection, data processing, data persistence, data consumption and integrating with ML workflow
• Ensure meeting the data governance and information security requirements
• Write unit/integration/functional tests
• Write technical specifications and provide inputs to docs team

Data Scientist

Deals with the data science specific aspects of the product/platform.

Expertise

• 5+ years of solid hands-on experience applying ML and statistical techniques to build models integrated into applications
• Deep knowledge of ML and statistical-analysis libraries, such as pandas, Torch, Caffe, TensorFlow, mxnet, scikit-learn, numpy etc.
• Experience of developing ML algorithms that can be scaled using multiple nodes
• Ability to explain and present core machine learning concepts to a broad business and technical audience
• PhD or masters degree preferred in Engineering, Computer Science, Mathematics, Computational Statistics, Operations Research, Machine Learning or related technical field or equivalent

Responsibilities

• Develop ML workflows that helps build, tune, deploy, version and trace ML models
• Create user-friendly & automated ways to implement sophisticated ML platform features e.g. ML features catalog, models management, hyper-parameter tuning, evaluation matrix etc.
• Design ways to improve visibility and explainability of model performance
• Write unit/integration/functional tests
• Write technical specifications and provide inputs to docs team

ML Platform Engineer

ML Platform engineers are responsible for developing ML platform tools and deployment automation aspects of the ML workflow such as CI/CD etc.

Expertise

• 5+ years of hands-on experience in writing and debugging ETL jobs using a distributed data framework (such as Spark, Kafka, or Flink)
• 5+ years of hands-on in scripting and CI/CD automation tools such as shell scripting, Ansible, Jenkins, Gitlab,
• 3+ years of hands-on experience with automating and deploying ML workflows
• 3+ years of hands-on experience with Kubernetes
• Experience with creating developer tools
• BS or higher in Computer Science (or equivalent)

Responsibilities

• Develop highly-available, scalable, flexible, and user-friendly ML infrastructure that enables rapid experimentation and production deployment of ML workflows
• Define build automation and packaging structures for ML model deployment process
• Create services and libraries that enable ML engineers to seamlessly transition from experimentation to production
• Pair with product teams and ML engineers
• Define security and backup procedures
• Write technical specifications and provide inputs to docs team

ML Ops Engineer

ML Ops Engineer handles operational aspects of the ML models in production

Expertise

• 5+ years of hands-on experience in container & ML pipeline technologies such as Kubernetes, Docker, serverless functions, Kubeflow, Jupyter
• 2+ years of hands-on experience using GPUs for accelerating ML training and inferencing workloads
• 2+ years of hands-on experience in implementing ML experiment tracking and monitoring architecture and tools
• Experience with model training and serving in production and at scale
• BS or higher in Computer Science (or equivalent)

Responsibilities

• Expose and securely operate product APIs for integration with 3rd-parties
• Make sure it is possible to monitor various metrics such as operation metrics, performance metrics and stability metrics from deployed ML models
• Design and provide the ability to scale and monitor models in production
• Provide the ability to deploy and track business level KPIs of multiple version of a model for the purpose of A/B testing
• Design and provide the ability to debug deployed models
• Build tracing, audit trails and observability frameworks for the deployed ML models on the ML platform