Software Engineer, LLM & Automation

About the role

The team builds automated pipelines for recruiting, finance, and project management using LLMs such as GPT and Claude. This role owns the design, implementation, and reliability of the data flow and control logic that connects these pipelines. You will define how information moves between third-party APIs, internal services, and language models to enable operational automation. The work requires a strong partnership with non-technical stakeholders to translate ambiguous requirements into precise system behavior. You will be responsible for writing production-level code that is testable, maintainable, and secure by default. A large part of the job involves debugging distributed workflows and ensuring that automated decisions remain consistent and explainable. Clear written and verbal communication is essential, as you will document designs and explain technical trade-offs to both engineers and business partners. This position is positioned at the intersection of software engineering and applied AI, where your technical choices directly shape how the organization operates.

Key facts

What you'll do

Architect and maintain automation pipelines that combine internal tools with LLMs, defining the flow of data and control for operational tasks across recruiting, finance, and project management.

Integrate data from third-party APIs such as ATS platforms, Slack, and Google into unified, automated workflows that operate reliably in production.

Design structured generation using JSON schemas and function calling to ensure robust and correct LLM outputs suitable for downstream automation.

Write and maintain integration code that handles concurrency, scaling, and performance optimizations in real-world deployments under varying load conditions.

Manage complex API orchestration, including handling rate limits, OAuth flows, error retries, and secure credential handling across multiple services.

Practice data-security best practices, including handling PII, encryption in transit and at rest, and secrets management to protect sensitive candidate and business information.

Collaborate closely with non-technical staff to gather requirements, clarify constraints, and ensure that implemented solutions align with real operational needs.

Approach problems with flexibility and creativity, adapting to iterative, experimental development cycles where requirements evolve through feedback.

Learn quickly about new developments in the LLM and AI ecosystem and apply this knowledge to keep solutions current, efficient, and effective over time.

Demonstrate strong communication skills by explaining technical decisions in plain language during planning, code review, and debugging sessions.

Spend part of each week on planning, code review, and debugging, ensuring that automated systems remain reliable and performant as usage grows.

Contribute to a culture of quality by writing tests, reviewing peer code, and improving existing automation to reduce technical debt.

Support on-call responsibilities and incident response for production pipelines, diagnosing issues and implementing fixes with minimal disruption to stakeholders.

Engage in continuous learning by experimenting with new prompting strategies, model capabilities, and tooling to improve the robustness of automated workflows.

Requirements

Program in Julia and/or Python with 3+ years of production-level coding, or demonstrate equivalent experience in other high-level programming languages.

Integrate LLMs using structured prompts, scaffolding frameworks, or advanced text generation approaches to achieve reliable and predictable behavior in automated systems.

Handle complex API orchestration, including managing rate limits, OAuth authentication, error retries, and secure storage of credentials across multiple third-party services.

Demonstrate the ability to work with JSON schemas, function calling, and structured data formats to ensure that LLM outputs can be consumed safely by downstream processes.

Apply data-security best practices, including techniques for handling PII, encryption, and secrets management, to maintain the confidentiality and integrity of sensitive information.

Strong collaboration and communication skills to gather requirements from non-technical staff and translate them into technical specifications and acceptance criteria.

Ability to learn quickly and adapt to new developments in the LLM and AI ecosystem, evaluating new tools and techniques to incorporate them where appropriate.

Approach problems with flexibility and creativity, thriving in iterative, experimental development cycles where solutions evolve through testing and feedback.

Comfort working in a fast-paced environment where priorities can shift and systems must remain stable under changing demands.

Experience with cloud infrastructure and deployment pipelines is beneficial for operating automated workflows at scale.

Willingness to practice extreme ownership by following code from idea through production, including monitoring, debugging, and iterative improvement.

Practical notes

Work arrangement is flexible, including contractors, part-time, or full-time options.

The role is based in the New York Office with possible hybrid expectations.

Typical interview steps

Hiring for engineering roles usually starts with a recruiter screen, followed by one or two technical rounds. Candidates often solve a coding problem, discuss past projects, and answer system design questions. Some loops include a take-home task. Final rounds typically cover team fit and give candidates a chance to ask questions. Interviewers look for how you break down an unfamiliar problem, not just whether you reach the answer. Practicing a few problems aloud and reviewing your own past projects are the best preparation.

Good to know

The role focuses on building reliable software systems that connect AI models with business processes. Core tools include LLMs, structured data formats, and cloud infrastructure. Success depends on clear communication, security awareness, and iterative experimentation.

Questions to ask

Useful questions for the interview: what a typical week looks like, how work is assigned, what tools the team uses, and how feedback works. Asking how the role has changed recently and what the team wishes it had known when joining is also reasonable. Questions about the manager's priorities are especially valued.

Career growth

Engineering careers usually progress from individual contributor to senior, staff, and principal levels. Some engineers move into management and lead teams of five to twenty people. Others stay on the technical track. Growth follows demonstrated impact, not tenure alone. A typical engineering ladder has clear levels with defined expectations for scope, quality, and mentorship. Moving up usually requires owning outcomes end to end rather than completing assigned tickets.