Outpost: A Platform for Deploying Internal Cloud Agents at PathAI

How we built a shared platform to give AI agents the internal context, tools, and environment to do real work across the organization.

By Harshith Padigela, Chintan Shah, Robert Egger, Harsha Pokkalla

At PathAI, AI agents have been an integral part of our software development and coding workflows for over two years. However, in the last six months, as model capabilities improved rapidly, we found ourselves delegating increasingly complex and long-running tasks to these agents and extending them into use cases beyond coding such as diagnosing platform issues, performing code reviews, and automating machine learning research and development workflows (1).

As these agents matured, we wanted to share their capabilities across the organization, including to technical and non-technical teams. We believe that as teams turn their workflows into reusable, agent-driven capabilities, they can become more effective while giving the teams that depend on them access to new tools and expertise. Workflows previously requiring a handoff can become self-serve. ML engineers can diagnose platform failures without looping in support engineers. Product and science teams can fine-tune machine learning models without relying on ML engineers for every iteration. We saw a similar shift across the industry, with teams moving agents beyond coding and chat into into broader use cases by connecting them to internal tools and workflows (2,3,4).

The Challenge

However, getting to this point meant solving a set of practical problems.

• Local agents don’t scale: Many engineers at PathAI leveraged local agents like Cursor, Claude, and Codex, customizing them with skills and tools for their workflows. These agents ran on individual laptops, meaning that any long-running work competed with other local tasks, and the agents stopped the moment the laptop was closed or lost connection. Sharing agents required recreating setup and credentials, which made them inaccessible to non-engineering teams.
• Cloud Agents have limited secure enterprise access: The cloud-hosted alternatives we evaluated ran on provider infrastructure and couldn't access PathAI's Virtual Private Cloud, leaving many internal systems like our platforms, clusters, proprietary data, and other tools out of reach.
• Democratizing agent development across enterprise: Building an agent from scratch meant dealing with access controls, deployment, session management and monitoring, barriers that limited access to non-technical teams. Our goal was to enable anyone to build agents by describing their workflows, leveraging their domain expertise rather than thinking about tooling and infrastructure.

During our annual Hackathon this past May, we reflected on how we could expand and standardize the use of agents across PathAI. The result was Outpost.

What is Outpost

Outpost is a platform for PathAI teams to build and deploy agents within our cloud infrastructure with governed access to approved internal systems and tools. It provides the runtime, credentialed access to tools, deployment, and interface so that teams can focus on what their agent should do rather than how to build and operate it.

Figure 1. Team members interact with agents in Slack, while agent owners define their instructions, skills and accessible tools. Outpost deploys and runs agents in PathAI’s compute cluster, connects them to internal systems and tools, streams updates, builds persistent memory and monitors agents over time.

Creating an agent. Teams define an agent by describing a role and providing operating instructions, including skills that encode specific procedures and evaluation criteria, trusted sources and templates as references, and any custom tools the workflow requires. Agents produce durable artifacts depending on the workflow, including files, code changes, trained models, documents, tickets or analysis. Authors can build and test an agent locally, and the same agent can be deployed through Outpost without additional infrastructure work.

Shared tools and infrastructure. Outpost runs agents inside PathAI’s cluster as Kubernetes workloads, allowing them to securely connect to approved internal systems. Each agent has a dedicated identity and policy-based access. For approved MCP integrations like Slack and Google Drive, agents access tools through dedicated proxy service rather than handling credentials directly. Outpost handles deployment, monitoring, and session management so teams can focus on the workflow.

Accessible through Slack. We chose Slack as the interaction layer as it is where most of our work already happens. It allows any team member to interact with agents from anywhere without any additional setup. Team members can tag an agent, describe the request, and continue the work in a thread. As work progresses, the agent streams updates back so teammates can follow along, steer, and share feedback.

Learning over time. As agents work, they write useful patterns, learnings, user feedback, and relevant project context into persistent memory. Future conversations build on those learnings rather than starting from scratch with each task.

What Outpost is enabling

Beyond its technical architecture, the real measure of the Outpost is the work that it enables. The following are a few workflows where teams are seeing the most value from Outpost today.

Improving machine learning models

Fine-tuning a machine learning model is an iterative experimental loop involving:

• Data preparation: identification of performance and data gaps,

• Labeling: collecting or annotating additional examples,

• Training: model training

• Evaluation: evaluating and interpreting model performance

• Iterate: deciding what parameters to change for model improvement.

Traditionally, each cycle has required ML engineers to manually run jobs, monitor progress, iterate on training and label collection loops, and coordinate across tools taking multiple days per cycle (at a minimum). The ML Agent now enables engineers to manage multiple models simultaneously, reducing their operational burden.

Through Outpost, technical and non-technical team members can now customize and trigger ML training and deployments directly from Slack using the ML Agent, without needing access to specific ML infrastructure tools or being dependent on ML engineers for each iteration. The agent coordinates data preparation, launches fine-tuning jobs, tracks progress, and surfaces evaluation results. Researchers and domain experts review the evaluations, interpret failure modes, and decide whether to accept the result or run another iteration. The agent maintains a durable project state, can monitor long running work, recover from failures and maintain context across different steps and iterations. The ML Agent is currently in an early pilot with a small number of projects. We are using this phase to validate the agentic workflow, improve the agent behavior, refine its guardrails and approvals, and tune the human-in-the-loop experience.

Figure 2. ML engineers use Outpost agents to run model-development loops spanning weeks. The agent has access to internal compute, data and experiment logging platforms and maintains a durable project database. Agents manage each stage of the lifecycle: splitting data for training and evaluation, collecting annotations, evaluating baseline models, and launching fine-tuning runs.

Other use cases

• Platform Support Resolution Agent: Internal platform support and on-call teams regularly investigate failures that span multiple systems including kubernetes clusters, logging and monitoring tools. With Outpost, the agent has access to the same tools and runbooks a support engineer uses, can collect evidence across systems, check whether a similar issue has come up before, and bring a diagnosis back into the Slack thread. Since launching this agent, the number of support requests handled by Outpost has grown three-fold, with agents now handling over 80% of our total support requests.

Figure 3. Percentage of support requests handled by Outpost agents in the 3 months since their introduction.

• Quality Documentation Agent: Much of product quality work at PathAI is structured assembly: verifying training completion, running periodic tool assessments, collecting supplier documentation, and drafting corrective action content per standard operating procedures (SOP). With Outpost, any team member can hand that assembly to an agent that knows PathAI's SOPs and document formats. The agent only produces a draft, which is then reviewed by our quality and regulatory team.
  
  

Where we are headed

Today, PathAI teams build agents on Outpost for their specific workflows, each with a clear purpose, tool set, and operating boundary. The result has been the swift incorporation of agents into our workflows to streamline tasks across functions.

Now we are working towards the next step: one primary agent that can serve teams across the organization. Rather than being limited to specific use-cases, this agent should be able to work on a broad set of complex goals potentially stretching across teams and running for days to weeks. Getting there requires agents to be able to understand the user’s goal, apply the appropriate access controls, coordinate work across multiple sub-agents and delegate long-running work to isolated compute environments. It also requires teams to define what success means, which evidence is trustworthy, what actions are allowed, and when the agent loop should stop, the emerging practice of loop engineering (5).

This is the future we are building toward with Outpost: agents that can handle complex tasks, working end-to-end across teams and over extended time horizons, all while remaining reliable, observable, and governed by the people who depend on them.

References

1. PathAI - Autonomous Agents for ML Research Acceleration https://www.pathai.com/blog/autonomous-agents-for-ml-research-acceleration

2. Background Agents (Ramp) https://builders.ramp.com/post/why-we-built-our-background-agent

3. Self-Driving Company (Replit) https://replit.com/blog/self-driving-company

4. AI-pilling Our Company (Sierra) https://sierra.ai/blog/ai-pilling-our-company-lessons-learned

5. Loop Engineering https://addyosmani.com/blog/loop-engineering/