Engenheiro de Dados e Plataforma de Agentes Sênior (GCP)

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Data Engineer & Agent Platform Engineer (GCP) – Engenheiro de Dados e Plataforma de Agentes Sênior (GCP) based in Brazil. As a Senior Data Engineer & Agent Platform Engineer, you will design and build scalable data infrastructure and AI-powered solutions within the Google Cloud Platform ecosystem. You will be responsible for developing reliable data pipelines, implementing data governance frameworks, and preparing high-quality data for production-grade AI applications. Working closely with data scientists, AI engineers, business teams, and external partners, you will help enable intelligent agents through secure data access, integrations, and modern agent architectures. You will also contribute to the development of RAG solutions, LLMOps pipelines, and agent-based systems using advanced Google Cloud technologies. Security, privacy, and regulatory compliance will be central to your work, particularly when handling sensitive healthcare information. This role combines data engineering, cloud architecture, AI infrastructure, and technical innovation in a collaborative, results-driven environment. It offers an opportunity to shape the foundations of next-generation AI solutions while delivering measurable business value. Accountabilities: • Design, develop, and maintain batch and streaming data ingestion, transformation, and quality pipelines using Google Cloud services such as BigQuery, Dataflow, Dataproc, Pub/Sub, and Cloud Storage. • Build and organize scalable Data Lake and Lakehouse architectures using BigQuery and BigLake, implementing appropriate data layers, partitioning strategies, and performance and cost optimizations. • Define and maintain data contracts, including schemas, data freshness requirements, and service-level agreements (SLAs), while monitoring data quality, consistency, and lineage. • Prepare and optimize datasets for AI applications through data cleaning, anonymization, chunking, and embedding generation to support Vertex AI RAG Engine, Vertex AI Search, and vector databases. • Ensure compliance with Brazil's General Data Protection Law (LGPD) when processing sensitive healthcare information, implementing masking, anonymization, Sensitive Data Protection, policy tags, and data residency controls before information reaches AI environments. • Design secure cross-project data access architectures using Cloud IAM, Dataplex, BigLake, and Analytics Hub, minimizing unnecessary data duplication while enforcing least-privilege access and auditability. • Enable internal systems and data sources to be consumed as tools by AI agents through MCP servers and MCP Toolbox, collaborating with AI teams on integrations with Vertex AI Agent Engine and Agent Development Kit (ADK). • Develop and maintain MLOps and LLMOps pipelines using Vertex AI Pipelines, incorporating model and response evaluations, security guardrails through Model Armor, and monitoring for data drift and response quality. • Provision and manage data and AI infrastructure as code using Terraform and CI/CD practices, ensuring consistent, automated, and scalable deployments. • Implement FinOps practices by monitoring and allocating infrastructure, project, and model costs through BigQuery billing exports and resource labels. • Establish end-to-end observability using OpenTelemetry, Cloud Logging, Cloud Monitoring, and Cloud Trace to monitor system health, performance, and reliability. • Contribute to resilience and disaster recovery strategies, ensuring the availability and continuity of critical data and AI services. • Collaborate with data scientists, AI engineering teams, business stakeholders, and external partners to communicate technical decisions, document architectures, and establish clear ownership of pipelines, data sources, and tools. Requirements: • Strong proficiency in Python and SQL, with the ability to develop efficient, scalable, and maintainable data engineering solutions. • Solid hands-on experience with Google Cloud Platform (GCP) data engineering services, particularly BigQuery, Cloud Storage, Dataflow or Dataproc, and Pub/Sub. • Experience designing, developing, and orchestrating data pipelines using Cloud Composer/Airflow, Cloud Workflows, or similar orchestration tools. • Knowledge of GCP data governance and security services, including Cloud IAM, Dataplex, policy tags, and Cloud KMS. • Practical experience with Terraform and CI/CD tools such as Cloud Build, GitHub Actions, or equivalent platforms. • Experience working with GCP environments involving multiple projects, organizations, and resource folders. • Strong understanding of distributed and event-driven architectures, including messaging systems, retries, idempotency, and data consistency. • Practical experience preparing and managing data for at least one production-level LLM application, such as Retrieval-Augmented Generation (RAG) or AI agents with tool-calling capabilities. • Knowledge of LGPD requirements for sensitive data processing, including data anonymization, masking, and privacy protection techniques. • Strong communication skills, with the ability to collaborate effectively with data scientists, AI engineers, business teams, and external stakeholders. • High standards of security, responsibility, and ethical judgment when handling sensitive information, particularly patient data accessed by autonomous AI agents. • Ability to evaluate and communicate technical trade-offs involving data duplication versus cross-project access, batch versus streaming architectures, and infrastructure costs versus latency. • Strong documentation habits and the ability to establish clear ownership and responsibilities across technical teams. Nice to have: • Experience with Vertex AI services, including RAG Engine, Vertex AI Search, Agent Engine, and Vertex AI Pipelines. • Familiarity w