Startup Job Board

Be part of the next big thing.
Explore career opportunities at innovative
startups in the U of T community.

Annotation Operations Lead - Modeling



London, UK
Posted on Tuesday, October 24, 2023
Who are we?
Cohere is focused on building and deploying large language model (LLM) AI into enterprises in a safe and responsible way that drives human productivity, and creates magical new ways to interact with technology and real business value. We’re a team of highly motivated and experienced engineers, innovators, and disruptors looking to change the face of technology.
Our goals are ambitious, but also concrete and practical. Cohere wants to fundamentally change how businesses operate, making everyone more productive and able to focus on doing better what they do best. Every day, our team breaks new ground, as we build transformational AI technology and products for enterprise and developers to harness the power of LLMs.
Cohere was founded by three global leaders in AI development, including our CEO, Aidan Gomez, who co-created the Transformer, which makes LLMs possible. Collectively, we're driven by the belief that our technology has the potential to revolutionize the way enterprises, their employees, and customers engage with technology through language.
Cohere’s broader research team is world-renowned, having contributed to the development of sentence transformers for semantic search, dynamic adversarial data collection and red teaming, and retrieval augmented generation, often referred to as “RAG,” among other technological breakthroughs.
We have been deliberate in assembling a team of operational leaders with industry-leading experience, with backgrounds working at the most sophisticated, demanding, and respected enterprises in the world. Cohere’s operational leaders have built, scaled, and led multi-billion product lines and businesses at Google, Apple, Rakuten, YouTube, AWS, and Cisco.
The Cohere team is a collective from all walks of life, from people who left college to start businesses, to some of the most experienced people from globally renowned companies. We believe a diverse team is the key to a safer, more responsible technology, and that different experiences and backgrounds enable us to tackle problems from all angles and avoid blindspots.
There’s no better time to play a role in defining the future of AI, and its impact on the world.
Why this role?
The Annotation Operations Lead is responsible for building and managing Cohere’s team of annotators and reviewers who develop and deliver high-quality training data critical to the development of Cohere's flagship retrieval-augmented generation (RAG) product. You will join a small and effective team who support Cohere’s data collection efforts.
Your day-to-day will include helping stakeholders translate their data needs into actionable tasks, running daily stand up and being the on the ground support for the annotators, working on projects to iteratively improve the way we run this program.

As an Operations Lead you will:

  • Scale the RAG teams annotation output while improving quality and effectiveness of their work
  • Partner with internal stakeholders to decide on staffing and hiring strategy to meet roadmap requirements, as well as work with external vendors to enrich our data pipeline as needed
  • Own individual and team-wide performance management of our RAG annotators, including partnering with the Data Ops team to continuously improve our performance management KPIs, plans, and processes
  • Run and develop onboarding for new RAG annotators, as well as identifying opportunities and tools for ongoing education to up-skill, coach, and improve team
  • Test-iterate regularly by running pilots and experiments, and stay up-to-date on industry trends and best practices
  • Assist with building library of high-quality prompts or prompt chains to accomplish a wide variety of tasks

You might be a good fit if you:

  • Have 2+ years experience leading a nimble team or complex project group before with great results in a tech-focused role
  • Are resourceful, focused, and highly-organized
  • Are known as an inclusive, effective and motivating communicator
  • Are ready to spend 4 days onsite each week in our London office with our Annotator team
  • Bonus points if you are an incredible writer - you are a word nerd, might love linguistics and are hyper-fixated on getting every comma in the right place
If some of the above doesn’t line up perfectly with your experience, we still encourage you to apply! If you consider yourself a thoughtful worker, a lifelong learner, and a kind and playful team member, Cohere is the place for you.
We value and celebrate diversity and strive to create an inclusive work environment for all. We welcome applicants of all kinds and are committed to providing an equal opportunity process. Cohere provides accessibility accommodations during the recruitment process. Should you require any accommodation, please let us know and we will work with you to meet your needs.
Our Perks:
🤝 An open and inclusive culture and work environment
🧑‍💻 Work closely with a team on the cutting edge of AI research
🍽 Free daily lunch
🦷 Full health and dental benefits, including a separate budget to take care of your mental health
🐣 100% Parental Leave top-up for 6 months for employees based in Canada, the US, and the UK
🎨 Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
🏙 Remote-flexible, offices in Toronto, Palo Alto, San-Francisco and London and co-working stipend
✈️ 6 weeks of vacation
Note: This post is co-authored by both Cohere humans and Cohere technology.