Director II, Data Science: GenAI Research and Solution

• Have a proven track record of working effectively with highly talented professionals to develop creative solutions to complex problems.
• Experience researching, fine-tuning, evaluating and communicating findings based on large and small language models.
• Experience designing and benchmarking GenAI pipelines such as RAG and corrective RAG, along with the evaluation methods that prove they work.
• Experience building and fine-tuning agentic orchestration — tool use, planning, memory and multi-agent workflows — along with the guardrails needed to trust it in production.
• Experience deploying and scaling GenAI solutions in production on a Kubernetes tech stack, preferably on Amazon Web Services (AWS).
• Be comfortable staying hands-on, writing production-quality code and reviewing the team’s work while leading it.
• Have an appreciation of the business applications and implications of scientific research.
• Have experience developing robust data and document processing pipelines at scale, including embedding generation and vector search.
• Have the ability to collaborate with researchers and engineers to bring complex ideas to market.

Solid communication and interpersonal skills are critical to the role, and the individual we hire will work very closely with their team, internal partners and external vendors as we work to test and implement new ideas. This is a highly visible position in a quickly growing area, where you will play a key role in building and growing the scope and impact of the GenAI solutions built by our team.

• Manage and develop a team of data scientists focused on GenAI research and solution delivery, promoting their technical and career growth along with their alignment with business outcomes.
• Set and drive the research agenda for incorporating GenAI deeper into Liberty Mutual products, from problem framing and prototyping through to measured business impact.
• Guide the fine-tuning, distillation and evaluation of small language models for domain-specific insurance tasks, balancing accuracy, latency and cost.
• Own the design and continuous improvement of GenAI pipelines, including RAG, corrective RAG and the retrieval and chunking strategies they depend on.
• Build and fine-tune agentic orchestration for insurance workflows, covering tool use, planning loops, memory and multi-agent patterns, and select the frameworks the team standardizes on.
• Stay hands-on with the work: prototype new ideas, contribute production-quality code and lead by example in design and code reviews.
• Establish rigorous evaluation and guardrail practices for GenAI use cases, including offline benchmarks, human-in-the-loop review and testing for groundedness, bias and hallucination.
• Productionize GenAI solutions on a Kubernetes tech stack, partnering with Engineering on containerization, autoscaling, GPU scheduling, observability and the continuous integration and continuous delivery (CI/CD) pipeline.
• Serve as a subject matter expert (SME) for GenAI based use cases at different stages of business.
• Develop and enhance MLOps and LLMOps best practices for data, prompt and model deployments.
• Partner with Engineering leaders to develop best-in-class processes for bringing complex GenAI features to market.
• Collaborate with third-party vendors and internal partners to evaluate new models, frameworks and data sources for GenAI use cases.
• Collaborate with Research, Data and other Science teams to define and deliver the datasets, document corpora and evaluation sets that the team’s work depends on.
• Partner with the Data Office and Data Steward to help prioritize and execute on complex projects, and to meet responsible AI and data governance standards.
• Present findings, share insights and make recommendations that impact profitability, growth and/or customer satisfaction.
• Provide tactical input on highly technically complex projects that drive change across function, SBU or Corporate Department.
• Supervise and perform highly complex, technical and creative data science projects.
• Regularly engage with the data science community and lead cross-functional working groups.

• Competencies typically acquired with Bachelors w/ 10+ years, Masters w/ 7+ years, or Ph. D with 5+ years of relevant experience in a quantitative field — preferably 2+ of those years in building and deploying generative AI or language model based solutions.
• Deep theoretical and practical understanding of transformer-based language models, including fine-tuning techniques such as parameter-efficient fine-tuning (LoRA/QLoRA), instruction tuning and model evaluation.
• Hands-on experience building GenAI pipelines with modern frameworks and tooling such as PyTorch, Hugging Face, LangChain or LlamaIndex and vector databases.
• Hands-on experience building agentic solutions with orchestration frameworks such as LangGraph or CrewAI, including tool calling, planning loops and agent evaluation.
• Demonstrated experience leading others through complex data-driven analyses and other technical work.
• Experience with Insurance industry products and business processes.
• Strong programming and MLOps skills are required, including reviewing and suggesting improvements to other team members’ code.
• Experience using a version control system such as Git.
• Experience building and maintaining a production-grade model pipeline using modern tools.
• Experience deploying and scaling containerized workloads with Docker and Kubernetes — including Helm, autoscaling and GPU-backed services.
• Experience interacting with Snowflake or other Cloud-based RDMS, and dashboarding using Datadog.
• Experience with building, maintaining and scaling pipelines with orchestration tools such as Airflow and or Luigi.
• Familiarity with CI/CD build pipelines and infrastructure-as-code is preferred.
• Expertise with Python and SQL.
• Ability to exchange ideas and convey complex information clearly and concisely, both verbally and in writing.
• Demonstrated ability to help move research projects from the idea stage into the development stage.