LLM Dev Analyst - Leadtech

We are looking for a highly skilled LLM Dev Analyst to design, build, and scale intelligent agents within our Databricks ecosystem, redefining how data is accessed, interpreted, and activated across the organization.

In this role, you will operate at the intersection of AI engineering, analytics, and data platforms, developing domain-specific agents that automate and augment decision-making across key business areas including A/B testing, marketing performance, data engineering workflows, and MarTech processes.

You will translate complex business challenges into agent-driven solutions powered by Large Language Models (LLMs), leveraging both structured and unstructured data to deliver real-time insights and automation at scale.

As a core contributor to our next-generation data stack, you will define and implement best practices for agent architecture, prompt engineering, evaluation frameworks, and orchestration within Databricks, ensuring production-grade reliability and business impact.

Key Responsibilities

1. Agent Design & Development

• Design, build, and deploy LLM-powered agents for multiple business domains (A/B testing, marketing analytics, data engineering automation, MarTech workflows)
• Develop multi-step reasoning agents that integrate with internal data systems, APIs, and tools
• Implement RAG architectures to enable agents to leverage enterprise data effectively

2. Databricks & Data Integration

• Integrate LLM agents within the Databricks lakehouse architecture
• Build scalable pipelines using PySpark, SQL, and Databricks workflows
• Enable seamless interaction between agents and data warehouses, event streams, and APIs

3. Prompt Engineering & Evaluation

• Design and optimize prompts, tools, and agent workflows for accuracy and performance
• Develop evaluation frameworks to measure agent quality, reliability, and business impact
• Implement strategies to reduce hallucinations and improve response consistency

4. Automation & Use Case Delivery

• Build agents that automate:
• Experimentation analysis (A/B testing insights)
• Marketing performance reporting and optimization
• Data Engineering workflows and monitoring
• MarTech processes and campaign operations
• Deliver solutions that drive measurable efficiency gains and decision velocity

5. Productionization & Reliability

• Productionize agent systems with monitoring, logging, and observability
• Implement guardrails, security controls, and governance frameworks
• Ensure scalability, latency optimization, and cost efficiency

6. Cross-Functional Collaboration

• Partner with Data Engineering, Analytics, Marketing, and Product teams
• Translate b