About the Role
You'll bring reinforcement learning to Firecrawl's core product — building the training infrastructure, reward pipelines, and fine-tuning systems that make our models meaningfully better at extracting, understanding, and structuring web data. This isn't theoretical RL research. You'll build your own training infra, run fast experiments, ship models to production, and bridge the gap between classical RL approaches and modern LLM agent systems. If you care as much about training throughput as you do about reward design, this is the role.
What You'll Do
- Build training infrastructure and reward pipelines from scratch. Design and operate the systems that train and evaluate Firecrawl's models. You'll own the full loop — data collection, reward modeling, training runs, evaluation, and deployment. You build the infra yourself because you're the one who needs it to work.
- Fine-tune models to achieve state-of-the-art results. Take foundation models and make them dramatically better at web data extraction, content understanding, and structured output generation. You know how to get from "decent fine-tune" to "best-in-class" and you have the patience and rigor to close that gap.
- Bridge LLM agents and classical RL. The most interesting problems at Firecrawl sit at the intersection of modern LLM-based agents and classical RL techniques. You'll design reward signals for agent behaviors, apply RL methods to improve multi-step agent workflows, and figure out where traditional RL approaches outperform prompting — and vice versa.
- Run fast experiments and iterate. You design experiments that test meaningful hypotheses, run them quickly, and make decisions based on results. You don't spend weeks on experiment infrastructure before getting a single result. Speed of iteration is a core part of how you work.
- Communicate clearly to non-RL people. RL can be opaque. You translate your work into language that engineers, product people, and leadership can understand and act on. You know how to explain why a reward function matters without requiring everyone to read the paper.
- Collaborate closely with the team. Work directly with the Search/IR-focused Research Engineer and the engineering team to connect RL improvements with search, ranking, and the broader product roadmap.
What We're Looking For
- Builds their own training infra and reward pipelines. You don't wait for an ML platform team to set things up. You build the training loops, reward models, data pipelines, and evaluation frameworks yourself — because you understand that infra choices directly affect the quality of results. You've operated GPU clusters, managed training runs, and debugged convergence issues in production.
- Can fine-tune models to SOTA. You've taken models from baseline to best-in-class on tasks that matter. You understand the full fine-tuning lifecycle — data curation, training dynamics, hyperparameter sensitivity, evaluation methodology — and you have the taste to know when a model is actually good versus when the eval is flattering.
- Bridges LLM agents and classical RL. You're fluent in both worlds. You understand PPO, RLHF, reward modeling, and policy optimization — and you understand how modern LLM agents work, where they fail, and how RL techniques make them better. You see connections between these domains that most people miss.
- Production-minded. You care about whether your models work in production, not just on benchmarks. You've deployed models that serve real traffic and made hard tradeoffs between model quality, latency, and cost. Research that doesn't ship isn't research that matters here.
- Runs fast experiments and communicates clearly. You'd rather run three rough experiments this week than one polished one next month. When you have results, anyone on the team can understand what they mean — no decoder ring required.
- Backgrounds that tend to do well: RL engineers at AI labs or applied ML teams who've shipped models to production. Researchers who've done RLHF or reward modeling for LLM systems. ML engineers who've built training infrastructure at startups and cared as much about the pipeline as the model. People who've worked at the intersection of RL and language models — whether in academic labs with a production bent or at companies building agent systems.
What We're NOT Looking For
- Pure theorists. If your best RL work lives in a paper and you've never trained a model on real data at real scale, this isn't the role. We need someone who builds and ships.
- Researchers who need a platform team. If you expect training infrastructure, data pipelines, and evaluation frameworks to be set up before you can be productive, you'll be frustrated here. You build the tools you need.
- People who only know one paradigm. Deep in classical RL but never worked with LLMs? LLM fine-tuner who's never touched RL? You'll be missing half the picture. This role requires fluency in both.
- Slow iterators. If your standard experiment cycle is measured in weeks, not days, you'll struggle with the pace. We need someone who can run a meaningful experiment, interpret results, and decide next steps within a day or two.
- Black-box communicators. If your typical update is a wall of metrics only another RL researcher can parse, this isn't the right fit. We need someone who can explain what's working, what's not, and why it matters — to people without RL PhDs.