Senior Lead Machine Learning Engineer
Company: Upwork
Location: Elkins Park
Posted on: January 20, 2026
|
|
|
Job Description:
Upwork ($UPWK) is the world’s human and AI-powered work
marketplace that connects businesses with highly skilled,
AI-enabled independent talent from across the globe. From
entrepreneurs to Fortune 100 enterprises, companies rely on
Upwork’s trusted platform and its mindful AI companion, Uma, to
find and hire expert talent, leverage AI-powered work solutions,
and drive business transformation. With on-demand access to
professionals spanning more than 10,000 skills across AI & machine
learning, software development, sales & marketing, customer
support, finance & accounting, and more, Upwork enables businesses
of all sizes to scale, innovate, and build agile teams for the age
of AI and beyond. Were looking for a Sr Lead MLE/Applied Scientist
to define how success is measured for AI agents performing
real-world tasks. This role is at the forefront of building trust
and quality into agentic systems by crafting rigorous, reproducible
evaluation frameworks that shape what we ship. You’ll work
cross-functionally to evaluate humanAI collaboration, assess
outcomes beyond accuracy metrics, and uncover what’s truly working
for freelancers and clients. Join us in revolutionizing agent
evaluation and making a measurable impact on AI systems that power
the future of work. Responsibilities • Design and implement
comprehensive evaluation frameworks that reflect real-world task
success for agentic systems, with a focus on humanAI collaboration
outcomes • Build benchmarking pipelines that capture nuanced
success indicators including trust calibration, intervention
frequency, and agent handoff quality • Lead development of
observability tools and instrumentation for analyzing agent
behavior in production • Translate complex qualitative and
quantitative signals into actionable insights that inform model
iteration and product prioritization • Collaborate with
researchers, engineers, and product teams to align evaluation
methodologies with business and user goals • Own benchmarking
infrastructure that enables reproducible, scalable evaluation
across AI initiatives • Champion rigorous experimental design and
statistical analysis across teams to ensure consistent and
meaningful measurement standards What it takes to catch our eye •
Proven experience designing evaluation systems for agentic or
LLM-based AI, ideally in complex, interactive or open-ended
environments • Deep expertise in statistical experimentation,
benchmark creation, and human-AI interaction assessment • Fluency
in building data pipelines and tooling using Python, SQL, and
distributed data processing frameworks • Demonstrated ability to
influence product and model roadmaps through evaluation insights
and performance measurement • Adaptive-level proficiency in
integrating AI tools into technical workflows for analysis,
experimentation, and observability refinement Come change how the
world works. At Upwork, you’ll shape the future of work for a
global, remote-first workforce, creating economic opportunities for
professionals worldwide. While we have a physical office in Palo
Alto, we currently hire full-time employees in 34 U.S. states,
making it easier than ever to join our mission from wherever you
call home. Our culture is built on trust, risk-taking, customer
focus, and excellence, all in service of our core mission: to
create economic opportunities so people have better lives. We
embrace authenticity and inclusion, encouraging everyone to bring
their whole selves to work. Personal and professional growth is a
priority here, supported through development programs, mentorship,
and our Upwork Belonging Communities.
Keywords: Upwork, North Bergen , Senior Lead Machine Learning Engineer, IT / Software / Systems , Elkins Park, New Jersey
